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ABSTRACT ^ 

Using the Cluster Analytic Model (CAM) , two samples 
of graduating seniors from Georgia state University were studied tc 
determine the effect o'; different patterns of college course work on 
their general learned abilities. The CAM uses Student Achievement 
Test scores, Graduate Record Examination scores, and transcripts for 
course work patterns. Growth in learning displayed in the test score 
results suggested that the sample students showed improvement in 
learning in analytic reasoning, reading comprehension, and 
quantitative comparisons. In the analysis of courses taken by five or 
more students, these three item-types explained large proportions of 
the score variance. Taking different course work produced different 
effects in general learned abilities, and those effects varied among 
the students of the two classes. Results also indicate: (1) that the 
CAM provides useful information to colleges about the mix of 
assessment measures that reflect what the students learn and what the 
college intends to teach them; (2) that it can assist at schools with 
a distributional general education requirement in assessment of their 
core curriculum; and (3) that it can, with computer linkages, help 
with student advising. The report includes a description of the 
research design for the model and a bibliography of over 200 items - 
(JB) 
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THE DIFyEREWTIAL COURSEWORK PftTTERNS PROJECT 

AHD THE DSVELOPWEHT OF THE CUISTER AHALYTIC MODEL 

The Differential Coursework Patterns (DCP) Project began in 1986 under 
Contract No. OERI-R-86'-0016 from the U.S. Departirent of Education and continues 
as a part of the research program in curriculum of the National Center for 
Postsecondary Teaching, Learning and Assessront. 

The purpose of the "Differential Coursework Patterns (DCP) Project" was to 
determine the effect of different patterns of college cours^iork on the general 
learned abilities of students. To acconplish this end, a model for linking what 
coursework students took in college with what they learned in college was devel- 
oped. The result was the Cluster Analytic Model • The Model groups courses ap- 
pearing on student transcripts according to the distribution of assessn»nt 
scores of those students. The Model uses precollege indicators of student 
learning to control for incosning student ability. It uses transcripts, rather 
than formal course or degree requir&nents as the representation of the college 
curriculum. The Model can use any nunft>er of assessnmnt iwasures, including both 
quantitative and qualitative data. 

Over the past five years, the DCP Project team has developed and tested the 
Cluster Analytic Model in a variety of college and university settings. Con-^ 
sistently, the Model has identified and correctly classified appraximately 8 of 
every 10 courses undergraduates took according to the multiple measures of stu- 
dent learning used. To date^ the Model has ©niJloyed such criterion imasures as 
the 9 item- types of the GRE General Test, Kolb's Learning Styles Inventory, a 
locally-^developed instrument measuring the students' perception of the diffi- 
culty of the courses, and the ACT CC»IP examination subscores. Discriminant 
analyses of the course groupings according to these measures indicated an 
average correct classification rate of over 80 percent. 

Sunroary of the Cluster Analytic Model . The basic steps in the DC? cluster 
analytic model are summarized as follows. For siJiplicity of exaii?)le, SAT scores 
were used as measures of inccming student ability. GRT item-type scores were 
used as measures of general learning. Coursework patterns were derived from the 
transcripts of graduating seniors at each of six postsecondary institutions. 
First, student achievement was determined by removing the predicted effect of 
SAT sub- scores on GRE item-type scores; the 9 item- type residuals served as 9 
criterion variables of general learned ability during the student's baccalaure- 
ate years. Second, the mean residual scores of students enrolling in each 
course were determined and attributed to that course. Third, courses were 
taxonomized (or grouped into patterns) using one of two cluster analysis 
procedures according to the 9 GRE residual scores of the students- Fourth, the 
secondary validity of the clustery and the contribution of each of the 9 
item- type residuals to the clusters were established by discriminant analysis. 
Fifth, the resulting patterns were described in terms of the cranbinations and 
sequences in which the students enrolled in them and the predominant curriculum 
characteristics of the courses in the resulting pattern. Lastly, the coursework 
patterns were examined to determine the extent to which they were associated 
with a discipline or set of disciplines and the general education requirements 
of the college or university. 
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Following these procedures produced a set of hypothesized relationships 
between coursework patterns and n»asures of general learned abilities. To pro- 
vide primary validation of these relationships^ a second sanple of graduating 
seniors was drawn from each participating institution, and the procedures of the 
cluster analytic siodel were repeated to determine the extent to which the 
coursework patterns and their relationship with the criterion measures of gen- 
eral learned ability were replicated. 

The conceptual framework for the cluster analytic model was initially de- 
rived from a review of selected literature. This review revealed that no one 
curricular model and no one analytical process clearly identified the effect of 
differential coursework on general learned abilities. Development and testing 
of an analytic model for studying the differential effect of coursework in a 
coii?>lex curricular organization was required, A conceptual-enpirical approach 
was adopted for the examination of the research prc*l«n^ wherein a conceptual 
f ran^work of student course decisions and selections guided the enpirical search 
for course%#ork patterns associated with gains in general learned abilities. 

The results of prior studies of IX3> student san^^les revealed substantial 
year-to-year variation in what students took (the coursevrork on their 
transcripts) and what they learned (the residual gains in student learning on 
the nine item-types of the GRE). Review of the data revealed that a substantial 
number of courses taken by the first saii?>le were also taken by the second 
san?)le. The analysis of cootoined sairples permits the further identification of 
coursework patterns beneficial to both groups while minimizing year-to-year 
variation in test scores. Given that the objects of analyses were courses and 
not students, scores from two consecutive student groups were combined for the 
purposes of further analysis of trends on coursework patterns. 

Findings of the Research . The following is an excerpt of the findings and 
conclusions from the combined first and second sai!9>les of graduating seniors at 
one of the institutions participating in the DCP Project^ Georgia State 
University (GSU). The sample discussed in this report consisted of native 
students who began and completed their education at Georgia State. A separate 
report (Ratcliff ,r 1991) presented the findings of a sainple of students who began 
at Clayton State College and subsequently transferred and graduated from Georgia 
State. Another report will be forthccHoing which focuses on the results of the 
total Georgia State san^Jle (the combination of both native and transfer 
students). The two student san?>les were similar to the population of graduating 
seniors for each year in terms of SAT scores r majors and deanographic 
characteristics. The two sainples were also similar to one another. There was 
no significant year-to-year variation in student characteristics, nor was there 
major variation in the characteristics of the sainples relative to the 
population. The san?>les were small and were cOTobined to permit further 
analysis. 

The growth in learning displayed in the test score results suggested that 
Georgia State Native students showed improvement in learning in Analytic 
Reasoning, Reading Ccmprehension, and Quantitative COTiparisons. In the analysis 
of courses taken by 5 or rrore students, these three item- types explained large 
proportions of the score variance. In each of these analyses, these GRE 
item- types proved to explain most of the gains in student learning. Taking dif^ 
ferent coursework produces different effects in general l^^orned abilities, and 
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those effects varied among the students of the 1986-87 and 1987 •-BS classes of 
graduating seniors « These results suggest that beyond growth in Analytic Reason- 
ing, Reading Coc^rehension^ and Quantitative Qjn^parisons, it is not neaningful 
to generalize abc^t the specific effects of GSU courseitork on the general 
learning of GSU Native students. 

What does the Cluster Analytic Model tell us about assessment? The Cluster 
l^alytic Model uses imiltiple s^asures of assessront. It provides colleges with 
information regarding the extent of variation in student assessment results that 
is e3q>lained by any one of the measures used. This information can be helpful 
in a number of ways* Faculty and administrators need not decide on p^i ideal set 
of assessment measures. The extent to which such erasures tmy overlap n 
describing student learning can be identified. The mix of asseesimnt measures 
appropriate to the goals of the college and the characteristics of the student 
population can be continuously monitored. When students show small amounts of 
growth on an indicator of student learning^ either the college can develop 
strategies for inproving student learning in the area identified^ or discard the 
measure as inappropriate to the college and its students. The Cluster Analytic 
Model provides useful information to the college about the mix of assessiront 
measures that reflects what the students learn and what the college intends to 
teach them. 

What does the Cluster Analytic Model tell us about the curriculum? The 
Cluster Analytic Model is a tool ideally suited to institutions of higher educa- 
tion with a distributional general education requirement and a wide array of 
programs^ electives and majors. For example^ if one of the assessn^nt measures 
a college selects is a test of analytic reasoning^ then the Cluster Analytic 
Model can identify those groups of courses that students took who shoved 
significant improvement in that area of general learning. Furthermore, the 
student population can be subdivided into high ability and low ability students, 
by gender, race or ethnicity, or by major. Then the Model can identify if the 
coursework associated with gains in learning among the total group is the same 
as that for the subgroups. Such appropriate information is valuable to 
curriculum planners. Courses in the general education sequence not found to be 
associated with gains in student learning can be revised, enhanced or dropped. 
Courses outside the general education requireronts that contribute to gains in 
student learning can becoi^ candidates for inclusion in the general education 
curriculum. The extent to which general education courses affect the learning 
of both high ability and low ability students has relevance in deciding how wide 
ranging the distributional options should be or whether a core curriculum is 
appropriate for the students and the educational goals of the institution. 

How can the Cluster Analytic Model help with advising students? By linking 
the coursework students take with their improven^nt in learning, the Cluster 
Analytic Model can be particularly valuable in advising students. First, it 
takes advising beyond the mere listing of formal degree requirements to the 
identification of those specific courses in which students of coaiparable 
interests, abilities and achieveiront have enrolled- Given several years of 
assessment data linked to the transcripts of graduating seniors, the Model may 
identify an array of courses taken by students who showed the largest gains in 
general learning in college. The Model is an^nable to the development of a 
microcoii55uter-based advising system utilizing a relational database of prior 
students coursetaking patterns and assessment results. Such a computer-based 
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advising system would yield an array of effective coursewrk tailored to the 
abilities and interests of individual students and within the paraaeters of in- 
stitutional degree requiren«nts . 
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INTRODUC?riO« 
The Differential Coursework Pa tterns Project 
and the Developroent of a Cluster Analytic W^el 



The purpose of the "Differential Coursework Patterns" (DCP) Project Lb to 
determine the effects of different patterns of college coursework on the general 
learned abilities of undergraduate students. Student san^sles *ere drawn as 
volunteer samples from 5 geographically and curricularly diverse colleges and 
universities; each saii?>le represented the diversity of Scholastic Aptitude Test 
(SAT) scores and majors of the graduating class at each institution. The 
precollege general learned abilities of each sanple of students was controlled 
in the research. The effect of SAT scores on the postcollege measures of 
student learning were determined and partialled from the analysis; the remaining 
residual scores were used as criterion variables of student achievement. TV.us, 
exiting student general learned abilities were operationally defined by the 
residual differences from the predicted and observed scores on 9 types of 
learning ("item-types") measured by the General Test of the Graduate Record 
Examination (GRE). 

The GRE General Test consists of three sections: verbal, quantitative and 
analytical; within each test section are specific types of test items (i.e. 
Verbal: analogy items; Quantitative: quantitative comparisons; Analytic: logical 
reasoning). There is a total of 9 item-types within the General Test, and the 9 
residual differences from the predicted and observed scores of students consti- 
tuted the primary measures of exiting student achievement in general learning. 
Thus defined, exiting student achievement served as a metric in the analysis of 
the differential coursework represented in the transcripts of each institutional 
sample. 



Six institutions of higher education participated in the original DCP 
project. Each institution represented a different Carnegie classification. The 
institutions were: Clayton State College (Morrow, GA: a public COTsnunity college 
at the time the enrollment of the san5>le); Evergreen State College {OlYii5>ia, WA: 
public liberal arts college II); Georgia State University (Atlanta, GA: 
doctoral-granting university); Ithaca College (Ithaca, NY: con?>rehenaive 
college); Mills College (Oakland, CA: private liberal arts I); Stanford 
University (Oakland, CA: private research university I). 

The approach taken in the DCP project was esgjirical. No a priori construct 
or model of curriculum was used to define what constitutes a differential 
coursework pattern. Rather, patterns were viewed as aggregationo of (a) 
individual courses , (b) canbinations of courses taken concurrently and (c> 
sequences of courses taken over tiii» . Thus, the smallest unit of analysis for a 
coursework pattern was a single course. For the purposes of analysis, each 
course examined had 9 attributes represented by the 9 residual item- type scores 
of students enrolling in the course. Courses with sufficient enrollment by the 
student sample were grouped according to the collective itean-type scores of the 
students enrolling in the course; thus, each course examined had a mean residual 
gain score for each item- type. 

The effect of individual courses, combinations of courses and sequences of 
courses on test score residuals was determined using hierarchical cluster anal- 
ysis. The secondary validity of the coursework clusters (patterns) was derived 
and their relationship with the 9 item-type residual scores was determined by 
dipcrirainant analysis. The resultant patterns were then examined in terms of 
their role in the formal general education structure of the institution, the 
dominant type of instruction represented in specific pat.:erns, and the nature of 



learning represented in those courses. 

The basic steps in the DCP clueter analytic model were as follows. 
Coursework patterns were derived from the transcripts of graduating seniors at 
each of six postsecondary institutions. First, student achievenent was 
determined by removing the predicted effect of SAT sub-scores on GRE iteo-type 
scores; the 9 item-type residuals served as 9 criterion variables ol general 
learned ability during the student's baccalaureate years. Second, the mean 
residual scores of students enrolling in each course was determined and 
attributed to that course. Third, courses were taxonomized (or grouped into 
patterns) using one of two cluster analysis procedures according to the 9 C»E 
residual scores of the students. Fourth, the secondary validity of the clusters 
and the contribution of each of the 9 item-type residuals to the clusters were 
established by discriminant analysis. Fifth the resulting patterns were 
described in terms of the combinations and sequences in which the students 
enrolled in them, and the predominant curriculum and instructional character- 
istics of the courses in the resulting pattern. Lastly, the coursework patterns 
were examined to determine the extent to which they were associated with a 
discipline or set of disciplines, the general education requirements of the 
college or university, the declared majors of the students enrolled, or the 
major denxjgraphic descriptors of the students (i.e., gender, race, age). 

Following these procedures produced a set of hypothesized relationships 
between coursework patterns and measures of general learned abilities. To 
provide primary validation of these relationships, a second sample of graduating 
seniors was drawn from each participating institution, and the procedures of the 
cluster analytic model were repeated to determine the extent to which the 
coursework patterns and their relationship with the criterion measures of 
general learned ability were replicated. The replication with a second student 



saiqple desnonstrated significant year-to-year variation in coursework and student 
learning. This reanalysis of Georgia State University students attempts to 
identify coimnon trends in learning and coursework between the two years. 

The conceptual framework for the cluster analytic model was initially 
derived frcsn a review of selected literature. This review revealed that no one 
curricular model and no one analytical process clearly identified the effect of 
differential coursework on general learned abilities. Development and testing 
of an analytic model for studying the differential effect of coursework in a 
ccarplex curricular organization was required. A conceptual-empirical approach 
was adopted for the examination of the research problem, wherein a conceptual 
framework of student course decisions and selections guided the rai^Jirical search 
for coursework patterns associated with gains in general learned abilities 

During the first project year (1986-87), a model was postulatei' to deter- 
mine effects associated with differential coursework patterns on selected mea- 
sures of general learned abilities of students. Requirements of this model were 
that it should have explanatory power regardless of institutional setting and 
should function independently of the particular assessment tool (i.e., the GRE 
exams) chosen. The generalizability of the model to other institutional 
settings was tested in the second year of the project. The model as initially 
postulated was refined using a supplemental historical data set of student 
transcripts and test scores from Georgia state University (GSU). The prelimin- 
ary testing and analysis of the model occurred later in the first project year, 
when the first sample of graduating seniors at GSU (Sample Group #1) was given 
the GRE and LSI instruments. Results from that administration were analyzed in 
previous DCP Project reports (Ratcliff, 1988, 1989), 

During February 1988, a random sample of graduating seniors at the other 
institutions participating in the study (Evergreen State, Ithaca, Mills and 



Stanford) were given the GRE and LSI. San5>le Group #2 at GSU was given the GRE, 
LSI and MPDQ in February 1989. Educational Testing Service scored the GRE 
tests, prepared a data tape of score results, emd retired and released the test 
forms to the five institutions tested in May 1988 for San^sle #1 and in May 1989 
for Sample #2. This report combines the findings frcai the native students at 
Georgia State University as San^jle Group #1 and Sainple Group #2. The findings 
for Georgia State San?)le #1 and Georgia State Sanple #2 were presented in 
earlier reports (Ratcliff, 1988, 1989, 1990). 

Organization of the Report 
This report is divided into 7 sections. Each section describes a major 
ccMT?x3nent of the research activity. Section 1 describes the theoretical basis 
for the differential coursework hypothesis, the developnent of the cluster 
analytic model as a means of testing it and the research objectives of the DCP 
px eject. Section 2 provides a detailed description of the procedures and 
methodology used in conducting the research. The cluster analytic model is 
described procedurally. Section 3 portrays the major characteristics of the 
combined sample #1 and #2 of graduating seniors whose transcripts and test 
scores were examined as part of the data gathering and analysis. Section 4 
describes the characteristics of the courses found on the student transcripts in 
terms of the criterion variables: the GRE item-type residuals. Section 5 
depicts the intended curriculum of the institution, its goals, curriculum 
organization and structure. Section 6 reports the findings from the combined 
sample using the quantitative cluster analytic procedures described in Section 
2. Section 7 offers a summary, conclusions and recommendations emanating from 
the research. 
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T^fcoearch Design for the Model 



'What are "general learned abilities" 

The cluster analytic model does not present nor does it rely on a 
particular theory of student learning. There is widespread disagreeinent on what 
constitutes "general learned abilities", and that disagreement is manifest in 
the variety of general education goals and degree requirements found in American 
higher education (Bergquist, Gould & Greenberg, 1981; Carnegie Foundation for 
the Advancement of Teaching, 1979; Gaff, 1983; Levine, 1978). within the term 
"general learned abilities", we mean to include such frequently used terms as 
"higher order intellectual processes" (Pascarella, 1985), "academic 
con5»etencies" (Warren, 1978), "generic con^tencies" (Ewens, 1979), "generic 
cognitive capabilities" (Woditsch, 1977), and "general academic ability" 
(Conrad, Trisman & Miller, 1977). Disagreement on terminology is but one aspect 
of the problems associated with measuring the general learning of students as 
undergraduates - 

Current notions of how to assess college outcomes call for multiple mea- 
sures of student achievement. No one measure has been found to accurately 
reflect the variety of definitions of general learning and cognitive develop- 
ment, the mixture of curricular goals and institutional characteristics found 
across the landscape of higher education and among the diversity of instruction- 
al procedures and curricular organizations of undergraduate higher education. 
The result has been a call for multiple measures of assessment of student learn- 
ing. Policymakers and academic leaders tend to believe that since colleges and 
universities have broad missions and goals, assessments should be congjarably 
broad enough to provide evaluation about as many institutional intents as 
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possible (Loacker, Cromwell & O'Brien, 1986; Nettles, 1987). 

Given the variety of terms, intents and theoretical frameworks used to 
explain •^general learned abilities", the research design presented here is 
criterion-referenced . That is, the design is not based on any one notion of 
what constitutes "general learned abilities" or any one college curricular 
structure intended to proniote student cognitive develofxi^nt. The design j^rmits 
the use of multiple and different measures of student outc(Mi»s, although the 
design is being developed and tested using one set of such measures. The 
validity of the assessment of student learning within this research design, 
then, is dependent upon the validity of the outcome measures selected, rather 
than the degree to which the student outcCTies suffice a global ( goal-free ) 
measure of student learning. The design fulfills the need for multiple measures 
and criterion-- referenced measures of student learning. However, because the 
design described below is not dependent upon any given college curricular 
structure or organization and, in fact, will be tested in five very different 
higher education institutions, it is free of bias engendered by specific 
institutional goals. 

What constitutes student achievCTient? 

Another question encountered in determining what constitutes general 
learned abilities is that of what ccffl9>oses the gains resulting from a college 
education. Singly measuring how graduating seniors perform on a series of tests 
is not a sufficient basis for generalizations about the effect of college on 
student achievement. First, the assessment of student outcomes is heavily 
affected by the students' academic achievement prior to entering college (Astin, 
1970a, 1970b; Bowen, 1977; Nickens, 1970). In fact, standardized tests used for 
college admission, such as the Scholastic Aptitude Test (SAT), have been shown 
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to be strongly correlated with tests used for graduate and professional school 
admissions, such as the General Tests of the Graduate Record Examination (ORE.). 
These correlations have been demonstrated for the total and sub-scores on the 
t%#o tests, suggesting that a large proportion of what postcollege tests, such as 
the GRE, measure are attributable to student learning prior to college, since 
the SAT and GRE examinations were used in the develc^sarent of the DC? cluster 
analytic model, further examination of their capacity to measure general learned 
abilities follows. 

Nichols (1964) studied the effects of different colleges on student ability 
as measured by the Graduate Record Examination's Aptitude Test. A sanple of 
356 National Merit finalists attending 91 colleges was used. The effects of 
college characteristics, major fields and types of colleges on student GRE 
scores were examined while controlling for the precollege student 
characteristics. Strong correlations existed between the students' SAT scores 
and the students' GRE scores which ranged from .65 for the verbal to .76 for the 
Quantitative. The results from this research indicated that the college 
students attended did have an effect on their GRE performance. There was a 
significant tendency for colleges to separate Verbal and Quantitative scores and 
raise one while lowering the other. Therefore, "the effect of college appears 
to be one of directing the students' abilities into verbal or quantitative 
channels rather than affecting the overall level of ability" (p. 52). 

Rock, centra and Linn (1970) and Rock, Baird and Linn (1972) examined SAT 
and GRE area test scores of 6,855 students who graduated from ninety-five col- 
leges, predominantly small, private liberal arts institutions. The correlation 
between college means on SAT-Verbal and GRE-Total was 0.91. However, the col- 
leges whose students had the same SAT means did not necessarily have similar GRE 
means. Rock, Baird and Linn found that "for colleges characterized by similar 
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ana relatively higher verbal input, the humanities data do suggest that propor- 
tion of faculty with the doctorate^ size of budget^ and selectivity are related 
to achieveTOnf (p. 15B). 

The fact that the standardized precollege and postcollege tests ^ such as 
the SAT and GRE^ are strongly correlated should not be surprising. Students 
typically bring 12 or more years of fomwl education with then upon entrance to 
college, and since the college years traditionally constitute 4 or 5 years, a 
large proportion of general learned abilities of students should be attributable 
to learning experiences prior to college admission « 

The strong relationship between the SAT and the CSRE is both an asset and a 
liability. The use of the SAT subscores as precollege nmasures and the GRE 
item-types as postcollege measures does provide a basis for controlling the 
effects of student acadanic achievement using comparable definitions of general 
learned abilities and cOT5>arable testing procedures. However^ the strong corre- 
lation between the two tests leaves only a small amount of explained variance 
between precollege and postcollege scores to attribute to general learning 
associated with a baccalaureate program. 

The dileimna posed by the use of the SAT and GRE as measures of general 
learned abilities among students is exacerbated by ths student population dif- 
ferences upon which the tests are norn^. Adelman *{ 1985) estimated that 25%-30% 
of graduating seniors take the GRE General examinations, while 60% of the grad- 
uating high school students take the ACTs or SATs. These rough percentages are 
understandable since fewer individuals choose to continue their education from 
bachelor's to graduate study than do those who choose to go to college frcsn high 
school. Nevertheless, a consequence is that the GREs are nonned on a higher 
ability population than the SATs {Pascarella, 1985). The individuals taking the 
GREs constitute a self ^selected sanyle, driven in part by the requirements of 
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graduate schools, professional schools, departments offering graduate degrees, 
and organizations requiring such examinations as part of the formal application 
for fellowships and scholarships (Adeljaan, 1985). In the DCP Project, a 
concerted effort was made to have each saB5>le of graduating seniors not limited 
to this self- selected group who would have taken the GRE as a normal event 
toward their application to graduate school. DCP Project sanpling procedures is 
discussed in Section 3. Nevertheless, it is ia^jortant to note that the Graduate 
Record Examination can be accurately viewed as a measure of a student's 
predicted performance in graduate school as well as a measure of that student's 
general academic accosfilishments as an undergraduate. 

The GRE and SAT tests have been criticized for a) the bias resulting from 
groups upon which they were normed (Adelman, 1985; Nettles, Thoeny & Gosman, 
1986) and b) their limitation in measuring higher order reasoning skills. These 
criticisms notwithstanding, the GRE and SAT tests do provide an economical, 
practical, and valid way of measuring selected general learned abilities while 
controlling for the incoming academic accomplishments of freshmen (Astin, 1968; 
Hendel, 1977). Critics of the GRE and SAT as measures of general learned 
abilities attack the validity of the measures themselves. These criticisms 
primarily are based on the use of sub-scores and total scores of the tests. The 
use of either the GRE or SAT item-type scores as multiple measures of general 
learning has previously not been widely explored (Adelinan, 1988), The DCP 
Project provided a detailed assessment of GRE item-types as discrete measures of 
general learned abilities. 

While the GRE and SAT tests were used in the development of the DCP cluster 
analytic model, it is not dependent upon these sets of measures. The model can 
be eitployed using another set of correlated precoUege and postcollege measures 
in a longitudinal analysis (for example see Pike, 1988.) For pragmatic and 
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economic reasons, the cluster analytic model was developed using SAT and GRE 
tests. The model was initially developed using the two sub-tests of the SAT and 
GRE: the SAT Verbal Test (SAT-V), the SAT Mathematical Test (SAT-M), the GRE 
Verbal (GRE-V), and the GRE Quantitative (GRE-Q). Subsequent develojsnent and 
testing of the model employed the 9 item-type parts of the GRE as multiple mea- 
sures of student learning. Therefore, the cluster analytic model will be des- 
cribed in terms of 9 measures of student general learned ability. 

Prior research suggested that the 9 item-type scores were independent 
measures of general learning. Wilson (1985) examined the criterion-validity of 
the 9 item-type part scores of the GRE General Test to the prediction of 
self-reported undergraduate grade point average (GPA). For his research, Wilson 
used the GRE scores of 9,375 examinees in 9 different fields of study 
representing 437 undergraduate departments from 149 colleges and universities. 
Data were first standardized within each department, then pooled for analysis by 
field of study. Results suggested that the GRE item-type scores did 
differentiate undergraduate GPA by field of study. This research and other 
studies (Powers, Swinton & Carlson, 1977; Swinton & Powers, 1982; Wilson, 1974) 
indicated that the 9 item- type subparts of the GRE measure different and 
somewhat unique general learned abilities. Initial testing of the DCP model at 
four colleges and universities also indicated that the item-types constituted 
independent measures of general learned abilities (Ratcliff, 1988, 1989). 

The GRE General Test consists of three sections: verbal, quantitative and 
analytical; within each test section are specific types of test items (i.e. 
Verbal: analogy item; Quantitative: quantitative con^Jarisons; Analytic: logical 
reasoning). There are 9 item-types within the General Test; the residual 
differences between the observed GRE scores (postcollege measure) and the GRE 
scores predicted by the students' corresponding SAT scores (precollege measure) 
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were used to gauge general learned abilities attributable to the students' 
undergraduate education. The residual differences between the observed and 
predicted values of each GRE it«n-type served as the 9 measures of student gains 
in general learned abilities during the time in which the student was enrolled 
in college. Thus in an economical and practical way, these student item- type 
residuals represented a set of multiple measures of general learned abilities 
which accounted for and controlled the effect of precollege student achievement 
on postcollege student outccanes. 

What constitutes a coursework pattern? 

The prevalent way to view the college curriculum is by its intentions, 
rather than by its results (Warren, 1975). Since measuring the effects of the 
curriculum is prcA>lematic, it is not surprising that many studies presume rather 
than test the effect of different patterns of coursework. 

The college curriculum is substantiative, additive and tenix>ral. In terms 
of cognitive theories ^f curriculum development, both content and process con- 
tribute to developmental learning in students (Tyler, 1950; Taba, 1962). 
Essentialist and constructionist theories of curriculum stress combinations of 
subjects (core curricula, great books, etc.) as influential on general learned 
abilities of college studentr (Fuhrmann and Grasha, 1983). The medieval univer- 
sity curriculum was organized according to combinations and sequences of courses 
as well as individual subjects (Rudolph, 1977); the seven liberal arts were 
sequenced into the prerequisite subjects of the quadriviuro (ewithmetic, geo- 
metry, astronomy, and music) and the higher order subjects, the trivium (logic, 
graimnar, and rhetoric). Together, the quadrivium and trivium provided an indi- 
vidual with the general learned abilities needed to study the three philosophies 
of Aristotle: natural philosophy (physics), moral philosophy (ethics), and men- 



tal phllosc^hy (jDetaphysics) . These coobinations and sequences of coursework 
have been generalized nore recently into concepts of breadth and depth as 
criteria by which to describe higher education curricula (Blackburn et al. , 
1976) . 

While the notion that combinations of concurrent coursework and devel- 
opmental sequences of course«i»rk lead to effects in the general learned abili- 
ties of students is derived from the medieval university, it is underscored and 
further supported by the research of conten^wrary develoini»ntal theorists. 
Perry (1968) for exaiif>le, stated that develq^nt "consists of an orderly pro- 
gression of cognition in which more conplex forms are created by the differenti- 
ation and reintegration of earlier, sin^ler forms" (p. 44). 

The value of ciirricular substance and sequence are presximed in formulations 
of core curricula, in the four levels of study (freshman, sophcarare, junior and 
senior years), in the corresponding practice of assigning course numbers accord- 
ing to those divisions, and in the practice of assigning course prerequisites. 
To assess the inqpact of these coursework patterns on the general learned abili- 
ties of students, the additive, substantiative and sequential characteristics of 
student course-taking need to be examined. These notions of what ought to be 
taught and what students ought to learn presumably represent the philosophical 
and educational aims of the particular college. 

Nevertheless, a distinction should be made between those patterns of 
coursework intended to fulfill undergraduate program and degree requirements and 
those patterns of coursework which students actually choose (Boyer & Ahlgren, 
1981, 1982, 1987; Warren, 1975). Intentional patterns of coursework are 
provided in a variety of publications issued by the institution: the college 
catalog, the annual schedule of times and days of courses, and program descrip- 
tions issued by departments and divisions within the college, Richardson et al. 



(1982) provide evidence that a minority of students may consult these stateAents 
of curricular intent prior to making decisions about which courses to choose. 
Other forms of intentional coursework patterns are the lists of courses or sub- 
jects required for certification or licensure in a particular profession, occu- 
pation or technical field. Such lists of coursework may be co^iled by practi- 
tioners and academics of a given discipline or profession to accredit college or 
university programs. Just as the curriculum of a particular college may repre- 
sent the philosophy and educational aims of that institution, so too may the 
certification, licensure and accrediting standards articulate the intentions of 
state, regional, disciplinary and programnatic asscx:iations . All are intended 
patterns of coursework in the ciirriculum whose measure of effectiveness, in 
part, is the extent to which these patterns accatplish their aims in practice. 

In a college curriculum, a single course may be the smallest unit of 
analysis. In assessing the inpact of the curriculum on the general cognitive 
development of students, the course constitutes a datum in the analysis. A 
pattern of data is a design resulting from "the relation anong a set of objects" 
(Romesburg, 1984, p. 278), In this case, the objects are courses. Therefore, a 
coursework pattern is a design resulting from relationships among courses. A 
cluster of courses (objects) is a set of one or more objects found to be similar 
to one another according to a given set of attributes. In the DCP Project, 
courses were grouped according to the extent of iiqproveanent (or decline) in 
general learning of the students enrolled. Thus, for the purposes of the DCP 
cluster analytic model, a cluster of courses is used to denote a pattern of 
coursework with an en5>irically derived set of relationships, stated another 
way, a cluster of courses is a pattern based on the actual enrollment of a 
cohort of students, rather than the intended enrollment pattern of the college 
or university or the set of courses any one student selected. This distinction 



its IsQportant in order to differentiate between the consequences of the college 
curriculum and its intents* 

Sources of d at a for coursework patterns 

Anjuments about what is and what is not an effective college curriculum are 
for the most part based on seasoned sf^eculation, nostalgia about academic tradi- 
tions, and unrealistic expectations of curricular coherence among and within the 
over 3/000 colleges and universities in the United States (Conrad, 1986). In 
most instances, the data used in describing the status of general education are 
derived from catalog studies and enrollment analyses. These data vmy not pre- 
sent an accurate picture of general education as it functions in students* 
programs . 

Catalog data provide evidence of trends in offerings but ignore student 
behavior. Catalogs indicate which courses a student should take to fulfill 
degree requirements and describe the intended contents or outcomes of the 
courses; collectively, this can5>endium of courses and articulation of degree and 
program requirements ideally leads to the accomplishment of the educational 
philosophy and curricular goals of the institution. White (1979) claimed that 
"college catalogs generally have little to do with reality", and that "the 
educational ideas expressed . ♦ . are subsequently neither perceived nor accOTi- 
plished" (p, 39). The ideals expressed are important, "yet when subjected to 
the rigorous scrutiny of time and experience, the academic promise is often not 
realized" (p, 39). White suggested that all collegiate programs, including 
general education, should be assessed in light of their original claims and 
promises , 

Dressel and DeLisle (1969) investigated college curriculum trends using 
catalogs as primary sources of information. They conceded that data derived 



from studies like theirs must be interpreted circumspectly^ asserting that: 

. . .mibiguities and contradictions arise in the use of catalogs for 
research of curriculum practices, because what appears in the catalog 
as policy is in reality often left to interpretation of individual 
advisors and individual dei^rtaents, and what is in reality required 
by an individual departjnent is in reality left to interpretation of 
individual advisors and what is in reality required by an individual 
department is often not stated as policy in the catalog ( p. 75). 

. . . the limits described by departments tend to be more exa^-ting and 
demanding than those stated in the institutional requireronts listed 
by the college. Thus, a question arises as to whether each student 
actually has flexibility and innovation claimed in the general state- 
ments or whether the department control of the major serves as a 
limiting and inhibiting factor in this respect (p. 78), 

Dressel and DeLisle also noted that catalogs, as sources of information about 
the curriculum, cannot be examined to determine the extent to which curriculum 
policies and stat«nents correspond to actual course-taking practices of stu- 
dents. Because of ambiguities, inaccuracies, discrepancies, and omiissions in 
wording in catalogs, the interpretation of the catalog varies significantly 
airong both students and advisors. Also, catalogs often do not present a ration- 
^ ale for course requirements, nor is there a way to determine how, why, or when 

requirements were introduced. Likewise, there is no way of determining "whether 
claimed articulation of liberal with professional education and of breadth with 
depth has been successfully achieved" (p. 79). In sum, catalogs do not provide 
information about the consequences of the coursework on student learning, and 
therefore, are not appropriate for learning outccaMs research. 

Assessments of the outcomes of college have little n»aning unless con?>ared 
with either a normative group of students or the intended curricular goals of 
the institutions. Catalogs are often the most conyrehensive statements of the 
intended substance and sequence of intended learning activities. From this 
standpoint, they provide a basis for con?>arison of college intentions with 
college outcomes. The DCP Project closely examined the extent to which the 
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college catalog represented the formal curriculum students experience in 
college. This cocf»rison is presented in Section 5 of this report. 

While college catalogs are inadequate in describing course-taking behavior 
of students, enrolliront figures as data sources also provide limited inform- 
ation. Although such figures are often used for evidence of curricular trends 
among undergraduates, enrollront analyses do not describe the actual course pat- 
terns of enrollees; however, they do reveal the extent to which different seg- 
ments of the college population choose particular majors or general education 
courses (intended coursework £mttems). Noting the inability of enrollB^nt 
analysis to determine student reasons for course selection, Friedlander (1979) 
used transcript analysis to examine the effect of changes in the coir5>osition of 
the community college student body on humanities enrollinents. Adelman, in an 
analysis of the Postsecondary Education Transcript Sanple (PETS) of the National 
Longitudinal Study of the High School Class of 1972, sunnnarized the advantages 
of transcripts as unobtrusive, en?>irical artifacts of student learning, 
"transcripts neither exaggerate nor forget" (1989, p. 1). 

Student transcripts as a data source 

Student transcripts are a rich, unobtrusive and problematic source of 
information about student course-taking behavior. Weurren (1975) used 
transcripts to determine coursework patterns among college students in a study 
of 50 history graduates of different four-year colleges. The student 
course- selection patterns in history, as revealed in these transcripts, indi- 
cated that within the discipline there were at least three or four different 
history programs. This finding demonstrated that although students receive 
similar degrees, they do not necessarily have the same educational ej^ieriences. 
Warren's study suggested that students shape their own curricula as they exer- 



cise options in choosing courses to coo?>lete credit hour requirements. Purther- 
nore, Warren danonstrated that transcripts could be used to discern broad 
curricular patterns. 

Blackburn and associates (1976) used transcript analysis in their investi- 
gation of curricular change and coursa-taking behavior in two and four year 
colleges and universities between 1967 and 1974. One of the goals of the study 
was "to determine how students utilize elective time" (p. 20). since some 
two-year transcripts did not indicate institutional requirements for both 
general education and the major, the researchers could not ascertain what 
courses were elective and which were prescribed- Consequently, the transcripts 
from two-year colleges were eliminated fron the sample used to describe stuoent 
course-taking patterns. The question of treatment of transfer students in 
transcript studies and college ijqjact studies has also been a consistent 
inethodological problem (Astin, 1970a). 

Prather and associates (1976) used transcript analysis to study undergrad- 
uate grading practices at Georgia state University and investigated differences 
in grading patterns by major fields of study while controlling for such ante- 
cedents as scholastic aptitude, demographic background, course types, and 
longitudinal trends. They found that major field was strongly associated with 
the grades students received in courses throughout the curriculum. This 
research and previous grade studies supported the proposition that the various 
parts of the curriculum have different grading standards, arguing against the 
use of GPA as a proxy measure of general learned abilities in college students. 
The studies by Warren (1975) and Blackburn and associates (1976) used small sam- 
ples of transcripts (5 percent) due to the time demanded in reading and assess- 
ing all the courses on each transcript. Prather and associates, however, used 
an electronic database of student transcript information to examine an insti- 
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tutional cohort; use of electronic databases of records enabled the researchers 
to examine larger samples of student records, thereby permitting analysis of 
larger sections of the ciirriculum. 

Transcript analysis has been used to examine the general education com- 
ponent of the undergraduate curriculum as well. The dean of instruction and 
curriculum planning at the University of Pennsylvania used transcript analysis 
in an effort to determine which courses among the many listed in the college 
catalog were actually selected by arts and sciences graduates (Carnegie 
Foundation, 1979). He found that 1976 graduates of arts and science programs 
had selected "a core of 29 courses" (p. 97) in the curriculum. However, not all 
students chose the same ctaribination of courses, and "many of the thousands of 
courses in the catalog that were not included in the core list were found on 
individual transcripts" (p. 97). This study illustrated one of the persistent 
problems in using transcript analysis to identify course-taking patterns: the 
enormous range of possibilities of course sequences generated by student choice 
in a large, multi-purpose university, it also suggested that, for whatever 
reason, there is a limited number of courses which most students select to 
con^Jlete the general education requireanents of the undergraduate program. 

Beeken (1982) used transcript analysis to examine the course-taking 
behavior of a sample of students in three Virginia ccwsminity colleges. The pur- 
pose of the study was to determine the number and types of general education 
courses selected by students to meet the general education requiren^nts of the 
Virginia Community College System. The study did not confirm the conclusion of 
the Carnegie Conroission that the general education curriculum was a "disaster 
area", although the programs of many students did not present a balance of 
disciplines; students apparently minimized the number of mathematics and sc.'.ence 
courses in their program of study. Both those who con?>leted an associate of 
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arts degree and those who did not exceeded the minlmnn requireronts for general 
education courses. The nuaber of courses taken in different curricular areas of 
general education were related to enroHiDent status, age, and sex. 

One of the largest collections of student transcripts is the Postsecondary 
Education Transcript Sanple <PETS) on the National Longitudinal Study of the 
High School Class on 1972 (NLS) . PETS data consisted of 22,600 students 
transcripts. While NLS has several precollege iroasures of achievement (high 
school grades, SAT, etc.) and the coursework selected by the students who 
attended college is represented, the NLS data has no available 
post -baccalaureate measure of general learning. Adeln»n (1989) used NLS/PETS 
and the NLS 5th Follow-Up Survey to demonstrate relationships between coursework 
taken in coBnnunity colleges and success in attaining bachelors and advanced 
degrees, career aspirations and plans, and self-reported attributes of the jobs 
the students held 15 years after high school graduation. In this analysis, 
transcripts proved to be a powerful, non-obtrusive measure of the relationship 
between what a student planned, what they studied at college, and what the 
nature of their work was a decade and a half later. 

In a limited number of studies, transcripts have been found to be a useful, 
valid and reliable source of information on student course-taking behavior. They 
provide evidence of the COTibination, sequence and perforjnance of students in the 
patterns of courses in which they enroll. As archival records, transcripts are 
unobtrusive data. While roost studies have limited their use of transcripts to a 
manual examination of a small san?)le of student records, there is evidence that 
such records, stored on a college or university computer, can be used to examine 
the course- taking behavior of a whole class, cohort or population of students. 
The cluster analytic model for determining the associated effects of coursework 
patterns on the general learned abilities of college students uses transcripts 
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in precisely this manner. Transcripts maintained on an electronic database can 
be merged with student score residuals for the purposes of assessing the effects 
of the curriculum on student learning. 

A conceptual f rairowork for analyzing coursework patterns 

The differential coursework hypotheL'is posits that individual courses on 
the student transcript contribute different levels of effect to the student's 
total college gain in general learned abilities (Benbow and Stanley, 1980, 1982, 
19B3; Pallas and Alexander, 1983). Thus, while all the courses Student X chose 
collectively affect X's gains in general learning^ the effects of an individual 
course on X's transcript n^y vary in its contribution to such an effect. 

For the purpose of analysis, the effects associated with a particular 
course are proxied by the residual scores of the students who enrolled in that 
course. A pattern of data is a design resulting frcxn "the relation among a set 
of objects'* (Romesburg, 1984, p. 278). In this case, the objects are courses. 
Therefore, a «?urseiiDrk pattern is defined here as a set of courses having 
ccsiyarable effect on one or more student score gains. The differential 
coursework hypothesis is rejected when no patterns are discernible among the 
data — when the score gains of students are uniformly attributable to enrollment 
in any and all courses in the curriculum. The hypothesis is affirmed when 
students who perform well on one or CK^re postcollege measures tend to enroll in 
certain courses and not others. 

At this point in the inquiry, the DCP cluster analytic model is not 
concerned with reasons for the effect. Rather, the next step is to identify and 
classify courses according to the score gains of students who enrolled in them, 
regardless of the factors that may have brought the students to enroll in the 
courses. Also, the sodel is not yet concerned with characteristics of the 
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students, although those characteristics may covary with the course selection or 
achiev^Mnt variables (Elton and Rose, 1967; Prather et al, 1976). Thus, in the 
examination of the effect of course patterns, there is no implication of direct 
causality of course patterns upon achievesnent (Astin, 1970a, 1970b). 

There is reason to presume chat the effect of a single course may vary 
according to what place it holds in the pattern of courses a student chooses 
(Prather et al., 1976). For exanple, if courses at a particular college are 
sequenced according to level (e-g., 100 level courses are intended for freshmen 
and 400 level courses are intended primarily for seniors), the effect of History 
101, **Survey of Western Civilization", may differ for Student X who enrolls as a 
first term freshmen froa Student Y who enrolls as a final term senior. Con- 
versely, logic holds that the effect of History 451, "20th Century American 
Foreign Policy", nmy differ for the first term freshman and the last term senior 
(Rudolph, 1977; Veysey, 1973). If a coiurse is viewed as contributing to the 
residual score for a particular measure of general learning, then a course's 
effect may vary according to its place in the student's pattern of courses. 
Therefore, course sequencing should be considered in the examination of course 
patterns (Bergquist et al., 1981). 

Likewise, the effect of a particular course may be associat'^d with the 
effect of other courses in which the student may be concurrently enrolled- 
Richardson et al. (1982) and Roueche and Snow (1977) noted that students may be 
advised to enroll in elementary writing or mathematics courses concurrently with 
other courseG requiring the basic skills these elementary courses teach. Under 
such practices, the student may have much less chance to succeed in college. 
Traditionally, the combination of courses in which a student enrolls within a 
given term is presumed to have effect (Bergquist, et al- 1981; Rudolph, 1977; 
Veysey, 1973) and therefore also should be considered in the analysis of course 




patterns. 

Thus, there is a great deal of uncertainty associated with why a particular 
student chooses a particular course at a particular time in his/her program of 
study. A poor grade in "Trigonometry" may cause Student Y to select a ranedial 
mathematics course over "Introduction to Calculus". Student X, who received a 
high grade in "Trigonoinetry" , may not enroll the following term in "Introduction 
to Calculus" because the time it is offered conflicts with a course student X is 
required to take within his/her major. Or the Calculus course may be filled 
when Student X tries to enroll. Many factors shape the cootoination of courses a 
student chooses in a given term and the sequence of courses represented across 
terms in the transcript. 

A modern research university may present 2,500 to 5,000 undergraduate 
courses fran which a student may choose 35 to 45 courses to ccsif)lete the bac- 
calaureate degree. Each semester or quarter a student enrolls, that student 
selects several courses. Each term of registration represents a stage in the 
overall decision-making process which generates the patterns of coursework found 
on the student transcripts at the time of graduation. Each enrollment decision 
is ILTiited and shaped by those courses in which the student has previously 
enrolled and the various degree requirements and prerequisites that are enforced 
during the registration process. At each successive decision-point, the student 
is progressively more immersed in the college environment, the norms and values 
of the student's peers, and the norms, values and expectations of the subjects 
the student selects to study. 

The analysis of the pattern of courses a student chooses is a sequential 
decision-making process wherein certain conditions exist: 

a. students make course selections in an environment of uncertainty 
about the consequences of their choices; 



- 18 - 



34 



2. there are sniltiple reasons why students enroll in each course; 

3. there are eiultlple options available to the student at each 
decision-point (term registration peric^); 

4. student course selections are seq;uential; there are different 
decision-points (terns) in which parts of the coursework pattern 
are chosen, with prior decisions having sane bearing on future 
decisions. 

Under the conditions listed above, students may choose courses to minimize 
uncertainty and risk (i.e., seek what they perceive to be "easy" courses). They 
imy also seek courses which will maximize the efficiency (i.e., fulfill degree 
and graduation requirements with a minimum amount of time), or maximize 
effectiveness (e.g., "it's a hard course, but I need to pass it if I'm going to 
major in engineering"). In this way, the succession of registration decisions 
ccmprising the student's pattern of coursework conceptually conforms to the 
multiple- stage decision analysis process (Buchanan, 1982; Bunn, 1984). 

According to Pace (1979), one variable in student development is the amount 
of time and effort invested by the student. This premise, that student involve- 
ment in learning advances student achiev«i»nt, guides the rectMranendations of the 
NIE Study Group's Report on Conditions of Excellence in American Higher 
Education (1984). Not only the kind and quality of cognitive activities in 
which the student engages, but also the level of effort exerted by the student 
in understanding and using the knowledge and abilities gained influence the 
quality of student learning. The student's effort in courses is "impressed" 
(Pace, 1979) by attitudes of the perceived usefulness of the course and the 
perceived difficulty of the course. These perceptions influence the kind and 
quality of student investment in learning. Coyne and Lazarus (1980) found that 
such investment involved both cognitive and subjective elranents, leading to 
whether the experience is viewed as a challenge or a threat. The perceived 
difficulty of courses influences student enrollment decisions and thereby 
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contributes to the TOiltiple-' stage enrollinent decision-jxmking process through 
which the student COTipiles his or her particular collection of courserork. 

In summary^ the literature suggests a number of possible interactions 
between student and cuurriculuin each tii!» a student makes course selections. The 
effect of courses on general learned abilities may vary according to the course 
itself, the time of enrollront in the student's baccalaureate progr^i, the 
conctirrent or sequential relationship to other courses in which the student 
enrolls, the predominant learning style of the course and of the student, the 
curricular design of the course, and the risk-taJcing behavior the student 
exhibits at each enrollment decision-point • The IX3> cluster analytic mouel 
calls first for the identification of student achievement (i.e., student score 
residuals), second for the classification of courses found on student 
transcripts into patterns according to their associated effects on the student 
score residual, and thirdly, for the further classification of courses within 
each identified pattern according to a) sequence and cwribination, b) the 
learning styles of the students, c) the risk-taking behavior of the students, 
represented by their perception of the difficulty of selected courses in which 
they enrolled, and d) the cOTsnon curricular characteristics of courses found 
within a given pattern of coursework* The model provides a basis for examining 
the extent to which the enqpirically^derived patterns of coursework reflect 
institutional mission and curricular goals, general educational requirements, 
the values, norms and mode of inquiry represented by the disciplines studied, 
and the demographic characteristics of the students. The model accomplishes 
these objectives through the use of cluster analysis, a statistical procedure 
which has been used throughout the physical and social sciences to derive 
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empirical taxonomies of objects in a variety of settings. Cluster analysis, 
since it has been infrequently ea^Jloyed in education, is described in greater 
detail in Section 2. 



Research objectives of the DCP Project 
The objectives of the DCP Project: 

1. To detennine student academic achieven»nt in general learned 
abilities gained during the baccalaureate program. This 
achievement was nwasured by the fixed criteria of the residual 
scores on the 9 itean-type subparts of the (^neral Test of the 
Graduate Record Examination (GRE), once the effects of precollege 
achievement as iroasured by Scholastic Aptitude Test (SAT) scores 
were removed. 

2. To classify the course"s*ork taken during a student's baccalaureate 
program according to its associated effects on the student's 
general learned abilities, as measured by the GRE. This coursework 
was determined by a cluster analysis of student transcripts wherein 
courses will be described and classified into patterns according to 
the GRE residuals of the students enrolling in them. 

3. To test the secondary validity of the coursework clusters and to 
identify outlying cases within each cluster of courses, 
discriminant analysis was applied to the results of the cluster 
analysis. Through examination of pooled within-group correlations 
of discriminant functions with GRE item-tyi^s and the cluster group 
means on each discriminant function, relationships between cluster 
groups and item-type residual scores was determined. 

4. To describe the resulting patterns of coursework according to: 

a- sequences and combinations of courses within the cluster, 
according to term of enrollment data found on student 
transcripts ; 

b. learning styles of the students enrolling; 

c. common curricular characteristics; 

d. perceived difficulty of the coursework, 

5. To determine the extent to which the resulting patterns of 
coursework are associated with: 

a. type of college or university; 

b. type of institutional general education degree requirements; 



c. type of academic discipline or field of study; 
d* student deiu>graphic characteristics. 

Saroles used in the Initial developront 
of the cluster analytic model 

For the puri»se8 of building and testing the cluster analytic mc^el, an 
historical database was developed at Georgia State Univereity- This database 
consisted of 1^024 students who began their baccalaureate education at GSU, 
graduated^ and then continued in graduate or profess ion^ikl education at GSU. To 
qualify for inclusion in this datal^e, a student must have cOTpleted 14 or more 
quarter credits at GSU. This same criteria also applies for Native State 
students. A student who completed more than one quarter (15 credits) at amother 
institution prior to enrolling at GSU was not included in the Historical Group. 
The database was drawn from all student transcripts at GSU between 1975 and 
1985. This population was selected because 1) it was a readily available 
database^ 2) prior research demonstrated that it was amenable to statistical 
analysis (Prather & Smith, 1976a, 1976b; Prather, Smith & Wadly, 1976), and 3) 
the transcript records contained SAT and GRE information as well as courses 
taken. 

FrcMn this database, 56 student records were found to contain the Scholastic 
Aptitude Test verbal scores (SAT-V), Scholastic Aptitude Test mathematical 
scores (SAT-M), Graduate Record Examination verbal score (GRE-V) and Graduate 
Record Examination quantitative score (GRE'-Q). All student identification 
information was removed from this database at GSU, so the individual identity of 
the student was unknown to the researchers developing the cluster analytic 
model. It should be noted that these 56 student transcripts were representative 
of the GSU student population in every way other than by major. Approximately 
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20% of the sasple were found to be psychology majors and another 20% were 
English majors. The spread of SAT scores appeared to otherwise approximate the 
d^nographic characteristics of GSU students. Therefore, the saaple did provide 
a database in order to develop the im^del- 

The historical database contained 1,024 transcripts upon which were listed 
an unduplicated count of 2,470 discrete course nuaobers and grades. The ample 
of 56 transcripts with cOT?>lete GRE and SAT test score information contained an 
unduplicated count of 1,065 discrete course nui^rs and grades. No evidence 
existed that a particular course offered in a particular year was indeed 
identical to a course bearing the sascm course nurr4>er in another year. The 
caQ(5>arability of courses in a cross-sectional study of a single cohort of 
college seniors may be less than that of a historical groups since the potential 
differences between courses bearing the san^ course identification nuntoer would 
only vary over about 4 years (from courses taken by the cohort when they were 
freshman to those con?>leted as seniors). A historical database accentuates the 
potential for significant changes in course structure, content or staffing- 
Nevertheless, for the purposes of im>del building and preliminary analysis, 
courses of the sasne course number taken in different semesters or years were 
assumed to be comparable. 

Courses repeated for credit were eliminated froax the analysis. For 
exanple^ MUS 101 was found to be a performance music class. One section of this 
class might be performance oboe, while another might be performance piano. 
Thus, students interested in music enrolled in multiple sections of the class 
during one term and enrolled repeatedly in the course over several terms. 
Likewise, HON 326 was found to be an honors seminar in the arts and humanities 
one quarter, in the social sciences the next quarter, and in the physical and 
life sciences yet another quarter. Therefore, these courses were eliminated 



from the analysis because they violatea the assuii?)tion of coo^jarability of 
course number over the quarters represented by the historical database. 



Figure 1-1. Transcripts in the GSU Historical Database 

TOTAL DATABASE SAMPLE 
N Percent N Percent 



Transcripts 1.024 100% 56 5.47% 

Unduplicated Course NunJoers 2,470 100% 1,065 43 12% 

Courses Taken By 5 or More 101 4.03% 
Students 



Prior research (i.e., Blackburn et al., 1976; Drees, 1982) used a five 
percent saitple of transcripts upon which to base generalizations regarding the 
undergraduate curriculum. The above analysis of the Historical Group database 
at GSU suggested that a 5 percent sample of transcripts would yield over 40% of 
the available courses in the college curriculum, but only about 5 percent of the 
curriculum will be represented by 5 or more students in the sairple. Therefore, 
generalizations about specific courses across a broad spectrum of the curriculum 
cannot be made based on the course-taking behavior of 5 or more students. Only 
those courses most frequently chosen by students may be included in such an 
analysis. Obviously, average class size has a bearing on the number cf courses 
available for analysis, given a specified transcript san^sle size and minimum 
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nuntoer of students required in each ccMirse cell in the cluster data matrix. An 
area needing further research is that of the relationship between sanple size of 
transcripts and representativeness of the curriculum as a whole. 

The relationship between 8aig>le size 
and the college curriculum 

A persistent problem in linking the undergraduate curriculum to n»asures of 
student learned abilities is the number of courses frcOT which students may 
choose. As previously mentioned, students will typically enroll in 35 to 55 
separate courses to ccHi5>lete their bachelor *6 degree^ although the nunft^r of 
courses vary considerably. Students select these 35 to 55 courses from a 
catalog of several thousand courses at a university or several hundred at a 
smaller college. Linking the effect of one course to the general learning of 
students therefore becOT^s problematic • 

Two sani>les of graduating seniors at Georgia State University (GSU) were 
drawn during 1986-87 and 1987-88. This report describes the results of the 
native students who began and con^leted their baccalaureate degree at GSU. 
However, since the individual san5>le sizes were small, a decision was made to 
coaiibine the seniles together resulting in a larger sanyle size to examine the 
important relationships. All of the data presented in this report concerns the 
combined sample. 

There were 7,850 total courses appearing on the transcripts of the 168 
students. After courses that were cross^listed or had equivalent numbers 
(through catalog changes) were identified; 1,244 unduplicated courses were 
counted on the 168 student course transcripts. Of the 1,244 courses, 300 were 
taken by 5 or more students in the Georgia State Native Group. The preliminary 
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analysis, therefore, focuses on these 300 courses (most frequently chosen by 
studencs) . 
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Figure l-2a. Sunmary of Georgia State Native Group 



USABI£ SAMPLE 
N Percent 



Transcripts, GRE Tests 

Duplicated Courses 

Unduplicated Courses 

Nunfloer of Courses Taken by 
5 or More Students 

Nuniber of Courses Taken by 
2 or More Students 



168 



7,850 



1,244 



300 24.12% 



704 56.59% 



The question of the representativeness of a saii5)le of courses to the total 
curriculum is exacerbated further by the lack of a precise definition of the 
total curriculum. As was evidenced by the analysis of Geor jia State Native 
student transcripts, the exact nurali^r of courses available to students does not 
correspond to the college catalog. Certain courses are offered in alternate 
years; others are offered less frequently. Scane courses are cancelled for lack 
of enrollment; others are split into multiple sections taught by different 
faculty due to large student demand. The exact number of unduplicated courses 
listed in the Georgia state curriculum database was not available to the DCP 
Project researchers. Likewise, the exact number of courses available for 
enrollment in any given year was not available. The courses in one year were 
not exactly identical to those offered the following year. What constitutes the 
curriculum, in terms of number of courses, content and variety, varies from term 
to term and year to year. 

Without an exact definition of the total curriculum available to under- 
graduates, the representativeness of a san^Jle of courses can only be approxi- 
mated. Since that definition of the curriculum evolves over the period encom- 



passed by a baccalatireate program, and since students enter and exit the bacca- 
laureate programs in different terms and the tenure of their undergraduate 
studies varies, the exact extent of courses frcxn which a student can make 
choices becomes individual, nebulous and isQsrecise. Even so, the data from the 
Georgia State native group reflects the extent of curricular change in even the 
most frequently enrolled courses over a six-year period generally covered by the 
transcripts. More research is needed in the variability of the course offerings 
on a yearly basis. 

In the DCP Project research, saB5>les of student transcripts were drawn and 
those courses enrolling 5 or nrare students were examined in the quantitative 
cluster analysis. When those courses enrolling 5 or nwre students were 
selected, the proportion of the total curriculum represented by those students 
was significantly decreased. When the initial sample size is not very large, 
the representativeness of the courses to the total curriculum may be seriously 
questioned. However, many debates regarding the vitality of the undergraduate 
curriculum in producing general learning among students consider only the 
general education portion of the curriculims, not every course listed in the 
catalog. From that standpoint, the representativeness of the courses included 
in the cluster analysis may be defined in terms of either (a) the total of 
courses offered during the period of enrollment of the student, or (b> the 
combinations and sequences of courses prescribed by the college or university to 
meet the general education requiren»nts for a bachelor's degree. 

The total courses offered during the period of enrollment of a student is 
not easily ascertained at many colleges and universities. First, the 
transcripts of a cohort of students list only what the students chose, not what 
was offered but they didn't chose, student choice of coursework is not nade in 
isolation, but is made in relation to those courses not selected, those 
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previously selected, and those planned for future terms. Second, not all 
courses offered during a given period are listed in the college catalog or 
bulletin. ExperiB»ntal courses and new courses, seme of which may be extended 
only in one year, do not ^^ar in the catalog. Coa5>aring the student 
transcripts with the college catalog reveals this. Thus, c-arses not listed in 
the catalog and not selected by a given cohort of students were among the range 
of enrollment choices available to the students. Lastly, there are courses in 
the college catalog which may not be given during the enrollment period of a 
cohort of students. While such courses were not choices to the student cohort, 
they were regarded as part of the formal curriculum of the institution. Thus, 
defining the curriculum as all courses available and/or advertised to a 
particular cohort of students may not produce an exact representation of the 
college curriculum. It may, in fact, obscure some of the roost e3q>erimental and 
innovative courses which, for one reason or another, did not get recorded in the 
college catalog. 

On the other hand, if one defines the curriculum pertinent to general 
learning solely in terms of what general education courses are required for 
degree con^^letion, the distinction between what the college intends and what the 
effects of the college curriculum are is blurred. The possibility looms large 
that a student enrolled in coursework that enhanced his or her general learned 
abilities but was not part of the formal general education requirements of the 
institution. Previously mentioned problems also exist with this definition of 
the curriculum as well: courses not selected are not fully represented, courses 
not listed in the catalog may be overlooked, and courses listed by not offered 
are treated as part of the range of options, m sum, the undergraduate 
curriculum is not a tidy item for analysis. 
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Por the purposes of the cluster analysis, the quantitative meaning of 
course attributes are more iasportant for generalization than the sanple repre- 
sentativeness. The effect of a given pattern of ccnirsework (its quantitative 
attributes, particularly) on a given student are more gemane to determining the 
differential effect of coursework than is the representativeness of that course 
to the total curriculum. If one wanted a sample of courses representative of 
all listed in the entire cuixiculum, one would need a large enough student 
sample so that the courses appearing on the transcripts of 5 or more »«mp^t, 
students would be representative of the total ciirriculum. This, again, presumes 
a means for determining the totality of a curriculum, given changes in courses 
offered over the period of student enrollnfflnt. It also requires an 
investigation of the relationship between the representativeness of courses to 
the curriculum and the relationship of the representativeness of the sanple 
transcripts to the student cohort or pc^lation studied. 

However, an alternate interpretation of the relationship between course at- 
tributes and their reliability would yield a different perspective on the above 
problem, in the initial analysis of the Georgia state Native group, the 
criterion variables used were correlation coefficients between SAT and GRE 
scores for those enrolling in a given course. Here, the probability of error 
varies according to the number of enrolled students. When GRE item-type score 
gains are used as course attributes, however, this does not seem to be the case. 
The residual score gains calculated among all students exhibited high confidence 
levels, as the regression functions of SAT scores on GRE item- type scores 
proved significant. The remaining concern, then, is the level of confidence 
attributable to a single course when it is detcribed by the mean of student 
residual scores from the san^sle group who enrolled in the course. 




At this juncture, it is inportant to note that the focus of the analysis is 
on courses, not students. It is not the purpose of the cluster analysis to pre- 
dict the population mean parameter of all the students enrolled in a course. 
Since the main ptirpose of the cluster analysis of college curriculum is to exam- 
ine the effect of an unknown course enrollment pattern on student general 
learned abilities, the confidentiality of mean residuals for a course is not of 
much iuixjrtance because the attributes are in large part significantly deter- 
mined by all students in the sample group, rather than by the students enrolled 
in the course. Thus, there is no reason for deleting those courses enrolling 4 
or less students from the cluster im>del building because the course attributes 
are determined by student course enrollment pattern, not by the characteristic 
of a single course. 

The analysis of the Georgia State Natives discussed later in this report 
revealed that each coursework cluster includes a certain variety of subjects and 
levels, ranging from those intended for freshmen (100 level courses) to those 
designed for seniors (400 level courses). These clusters, derived of courses 
sorted according to the gains in general learned abilities of the students who 
took them, generate questions regarding the basic attributes of the college 
curriculum: discipline, sequence and level. 

Assumptions 

1. One learns what one studies. 

2. Courses are the primary units of learning in college. 

3. Transcripts are an accurate listing of the enrollment pattern of 
students . 

4. Most undergraduate courses are basically stable in content and 
instruction over time and among instructors. 
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5. The effects measured can be generalized to all the formal 
ccHirsework in which a student enrolled. 

Limitation in analysis of curricular ^t terns 

The analysis of coursework patterns that lead to higher student gains in 
general learned abilities should provide a description of the effect of 
different aspects of the undergraduate curriculum. The analysis should also 
point to those curricular characteristics and organizations which promise to be 
most effective for prOTioting cognitive develc^sment. However^ the analysis is 
delimited to those students who were attaining the bachelor *s degree at Georgia 
State University and the other institutions participating in the DCP Project. 
No analysis is presented of the differential effect of coursework on those 
students who ended their studies prior to cop^pletion of their senior year. 

Two forms of error need to be avoided in such an investigation. One is the 
reductionist error of attenpting to account for the variance in cciiplex groups 
ings of coursework through individual psychological variables. In a recent 
review of the literature on the eftects of race, gender and class on educational 
attainment. Grant and Sleeter (1986) concluded that attention to one status 
group oversimplified the analysis of student behavior, confirming the problems 
of reductionist error. The reductionist error may also occur in research equat- 
ing general learned abilities within con^slex academic organizations with 
intra-group cohesion and/or with individuals' identification with an academic 
discipline. The study of student learnii g in colleges and universities is a 
study of student behavior in such organizations, rather than the study of such 
organizations . 

A second type of error is the unigueness-of-data approach . Vihile it is 
iinportant to acknowledge what is unique in each institutional learning 
environment, this should not halt the exploration of appropriate relationships 
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of curricula within different colleges and universities. This error may emanate 
from the failure to conceive of these institutions as systems (1) nested within 
and linked to larger systems (disciplinary and professional fields), and (2) 
containing snaller subsysteiM (departments, divisions and programs) that are, in 
turn, linked to them (Katz, Kahn and Stacey, 1982). 

Prior research suggests that student coursework patterns found to affect 
general learned abilities can be characterized by (1) the extraneous (other than 
achievement) characteristics of the students enrolled, (2) the unique or idio- 
graphic characteristics of the learning environment, and (3) the normative 
effect of the fields of study on learning in colleges and universities (Astin, 
1970a; Pascarella, 1985) , Prior research has also demonstrated that more than 
one model of college curriculum can explain the effect on student laarning from 
a conmon set of transcript data (e.g., Hesseldenz and Smith, 1977; Kolb, 1973). 
Therefore the cluster analytic model identifies eB?>irically-derived course 
patterns which subsequently may be examined in terms of st'^dent characteristics 
and Idiographic and nomothetic aspects of the curriculum. In this sense, the 
cluster-analytic model is retro- deductive in approach and is useful to the 
generation of research questions and hypotheses regarding ccanmon notions of the 
college curriculum and its relationship to general student learning at the 
undergraduate level. 
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Definition of terms and concepts 



Transfer student ; A student who has earned the equivalent of one or more ternis 
of full-time work (15 quarter credits or 10 sea»ster credits) at another 
institution of higher education prior to enrolling at the institution under 
study. 

fative student; A native student has (A>tained his/her undergraduate educational 
e3q>erience prijnarily from the institutioi, under study. Native students 
entered the college or university with no more than 14 quarter credits or 9 
8«nester credits earned at other institutions. Native students may have 
accumulated coursework at other institutions during their bachelor's 
program (i.e., a student who attends sunaner session at another institution 
during her junior year), but such credit does not constitute a significant 
portion of their baccalaureate program. 

Graduating senior; A student who has declared his/her intention to graduate or 
is estimated to graduate during the calendar year ccmsnencing July 1st 
andending June 30th. 

Scholastic Aptitud e Test {SAT) Scores ; Students may have taken the SAT examina- 
tions more than once prior to admission. VJhen more than one set of SAT 
scores was available for a given student, the SAT score date inanediately 
preceding the initiation of the baccalaureate program was used. That is, 
if a student took the exam several times and entered college in September 
1980, then the SAT scores frcai the test most ianrediately preceding 
September 1980 were used. The SAT is a precollege effects measure, and the 



ERIC 



fliore proxlmous the oMasure is to the initiation of the effects to be 
analyzed^ the xssDst desirable. 

Quarter calendar ; The calendar usually consists of four ten-week terms. 

Seroster calendar ; The calendar usually consists of tw terms which average 
fifteen weeks each. However 4^ each term can be as long as twenty weeks. 

Description of the GRE General Test 

The GRE General Test purports to measure verbal^ quantitative and analytic 
abilities iii?>ortant to acad^uic achievement (Educational Testing Service^ 
1988). In doing so, the test reflects the opportunities and efforts of the 
student to acquire these abilities. 

Verbal abilities (GRE-V). One of the major subscores of the GRE General 

Test is that of verbal ability. Verbal ability is described as the ability to 

reason with words in solving problems. 

Reasoning effectively in a verbal medium depends u|K>n the ability 
to discern/ CCTjprehend and analyze relationships among words or groups 
of words and within larger units of discourse such as sentences and 
written passages. Such factors as knowledge of words and practice in 
reading will. . .define the limits within which one can reason using 
these tools (ETS^ 1988, p. 28). 

The GRE Verbal Subscore is derived frcan 4 types of items: analogies, 
antonyms, sentence coit^Jletion and reading connprehension questions. Each is 
described below: 

Analogies (ANA). Analogy items test students' ability "to recognize 
relationships among words and the concepts they represent and to recognize when 
these relationships are parallel. The process of elijDinating four wrong answer 
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choices requires one to fomnilate and then analyze the relationships linking six 
pairs of words" (ETS, 1988^ p. 28) • 

Antonyms (ANT). Antonym itcans provide a direct test of the student's 
vocabulary. However, the purpose of this it«n-type is not merely to measure the 
student's vocabulary, but also to gage "the student's ability to reason frooi a 
given concept to its opposite" (ETS, 1988, p. 29). 

Reading Ccanprehension (RD). To successfully cojcplete these it«ns, students 
must read narrative with "understanding, insight and discrimination". These 
passages challenge a student's ability to analyze using a variety of 
perspectives "including the ability to recognize both eaqplicitly stated elements 
in the passage and assumptions iinderlying stat^ients or argun^nts in the passage 
as well as the inplications of those stateronts or arguments" (ETS, 1988/ p. 
31). Due to the length of the narr::/wives around which the questions for this 
item- type are built, students are given an5>le opportunity to assess a variety of 
relationships, such as the function of a key word in a passage, the 
relationships among several ideas, or the relationship of the author to the 
topic or the audience. 

Sentence Cong^letion (SC). These items determine the student's ability to 
"recognize words or phrases that both lexically and stylistically COTplete the 
meaning of a sentence" (ETS, 1988, p. 30). The student must decide which of 
five words ^ sets of words or phrases can best complete a sentence. In 
completing this type of task, the student must consider which answer gives the 
sentence a logically satisfying meaning and stylistically integrated whole to 
the discourse. 

Quantitative Ability (GRE-Q). The second subscore of the GRE General Test 
measures basic mathematical abilities, the understanding of basic roathmatical 
concepts, the ability to reason quantitatively and to solve problems that 



- 38 - 



require skills in mathd&atical analysis. The quantitative itesns seek not to 
exceed the abilities cossraon to undergraduates, regardless of field of study. 
Questions test the student's facility with arithrotic^ algebra and geometry. 
Questions may be in words, metric units and synftiols or figures, graphs and 
tables. 

Regular Mathanatics (RM), This quantitative it«i-type has also been 
labelled Discrete Quantitative questions and Arithmetic, illgebra and Geooietry in 
various GRE and ETS publications. 

Quantitative CoBgarisons (QC), These iteaos test the student's ability "to 
reason quickly and accurately about the relative sizes of two quantities or to 
perceive that not enough information is provided to make such a decision" (ETS, 
1988, p, 34). 

Data Interpretation (DI). Data interpretation items presents sets of data 
in graphs and tables and ask students to synthesize the information, choose the 
correct data to answer the question, or to determine that the information net >d 
is not present in the data set* 

Analytic Ability (GRE-A), The third subscore of the GRE General Test it 
designed to measure students* ability to think analytically. This subscore - 
ccsnprised of two item-types: Analytic Reasoning and Logical Reasoning- 

Analj^tic Reasoning (ARE)- Analytic reasoning items measure a student's 
ability "to understand a given structure of arbitrary relationships among 
fictitious persons, places, things, or events, and to deduce new information 
from the relationships** (ETS, 1988, p- 38). 

Logical Reasoning (LR). These items assess a student's ability to 
understand, analyze and evaluate positions and contentions. Specific questions 
may evaluate a student ability to recogiiize a point of argument or the 
assumptions on which a position is ba;^ecl, to draw conclusions or form 
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hypotheses, to assess the manner of argusiGnts and the evidence si^^rtlng them. 

Ilhile the GRE General Tests are designed to describe the student's brc»d 
verbal^ iratheKmtics and analytic abilities < the 9 individual item- types of the 
Test provide discrete measures of general learned abilities. One should avoids 
however, making the assumption that the GRE i^asures all general learned 
abilities associated with collegiate learning or even those intended as the 
educational goals of a particular college, university, program, major or 
course. Nevertheless, the GRE provides a broad set of n^asures of general 
learning from which a n^el to assess selected gains in cognitive development of 
college students. 

S\immarv 

In this section the purpose, jcope, and n^thod of the DCP Project were 
described. The purpose of the research was to test the hypothesis that student 
enrollment in different patterns of coursework affects the development of their 
general learned abilities - 

There was no clear consensus in the literature reviewed as to what 
constitutes the general learned abilities of college students. There was 
general agreenmnt that the assessment of students' general cognitive development 
necessitates multiple iwasures of general learning. 

One set of such measures consists of the 9 item-types of the general test 
of the Graduate Record Examination. The GRE General Test has been criticized as 
an assessment measure of baccalaureate learning because of the limited nuntoer 
and advanced abilities of the students upon which the tests were normed. 
Similarly, the multiple choice format of the GRE has been criticized for not 
measuring higher-order reasoning and creative thinking skills. These criticisms 
notwithstanding, the GRE presents a ccsimion set of measures of general collegiate 



learning. 

The strong correlation of the GRE with the Scholastic Apptitude Test (SAT) 
affords an WJrtunity to control the assessment of student learning for the 
coofiarable kncntledge^ skills and abilities students possessed upon admission to 
college. The largest amount of variance in general learned abilities logically 
should be attributable to learning prior to college. Nevertheless, when such 
learning is removed from the analyses, residual scores should provide indicators 
of students' develc^w^nt during the college years. 

Just as deterroining what constitutes student achievement in college is 
problematic, so too is the determination of what the formal curriculum of a 
college or university is. A distinction in the DCP Project is made between the 
intended curriculum, as stated in the college catalogs and bulletins and the 
actual curricular record, as represented on the student transcript. A 
coursework pattern was defined as a set of courses whose effects on general 
learning are similar. This definition is an empirical artifact of student 
enroUjinent behavior, rather than an academic plan or stated curricular sequence, 
as might be found in a catalog or program brochure. 

By clustering courses into patterns according to their effect on general 
learned abilities, a basis is provided for examining what students took in light 
of what they learned. If what the GRE measures was what the college intended as 
the outcomes of the general education curriculum, and if the course shown to 
afiect positively the general learning of undergraduates were the same as the 
colleges general education requirements, then that college may take pride in the 
evidence of the effectiveness of its curriculum. But such a comparison of what 
a student takes with what that individual learns is predicated upon the 
hypothesis that enrollment in a different pattern of coursework leads to 




different effects in general learning. 

To test the differential coursework hypothesis, a cluster analytic model 
was developed. The residual scores of GRE itean- types were attributed to the 
specific coursework in which students enrolled. Each course was described in 
terms of the mean of residuals of the students who had enrolled in the course* 
Thus, there were 9 mean item- type residuals for each course found on student 
transcripts. Next, courses were sorted and clustered according to these 9 
criterion variables. Finally, the validity of the groi^ings was tested using 
discriminant analyses. From the discriminant analyses, coursework affecting 
general learning could be differentiated from that serving sam other role (such 
as learning within the major or learning not ii»asured by the GRE). 

Through the developa^nt of the cluster analytic model, the effect of 
coursework on general learned abilities may be assessed* Furthermore, the 
extent to which the item-types of the GRE represent discrete measures of general 
learning can be assessed* Similarly, the intentions of a college or university 
general education curriculum can be compared to empirically-derived courseirark 
clusters found to \^ associated with gains in general learning. Thus, the model 
can be used to assess student learning to determine the strength and 
independence of the measures of learning selected, and to ccrapare the intended 
curriculum with the actual course- taking behavior of students. 
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TI. Methodology and Procedures: 
Cluster ftjialytic Model 

A conceptual/ 6iiQ>ir leal approach was used in the selection, testing and 
adoption of a specific methodology for the analysis of coursework pattexms- 
The approach was conceptual in that theoretical concepts differentiating 
coursework discussed in the previous section restricted the enpirical approach 
to conform to the nature and orientation of the research problem. What follows 
is a discussion of the process of cluster analysis and its application to the 
investigation of coursework patterns; in that discussion, cluster analysis is 
contrasted to other statistical methods of potential value to the research 
investigation- 

Previous transcript analysis studies have used the gt^neral linear model and 
regression analysis (Benbow and Stanley, 1980, 1982; Pallas and Alexander, 1983; 
Prather and Smith, 1976a, 1976b). The rationale for the use of regression is 
based upon practical and theoretical justifications* Regression analysis allows 
maximum design flexibility and is statistically robust. Transcript analyses 
involve large amounts of data. For exanqjle, Prather et al. (1976) examined 
8,735 student transcripts which collectively contained 189,013 individual course 
grades. Regression analysis provides an effective technique for presenting the 
diverse nature of the data while maintaining a consistent analysis rationale. 
However, the general linear nK)del does not provide a direct means of assessing 
the additive and temporal aspects of course patterns, as described in the pre- 
vious chapter. 

To distinguish cluster analysis frcmi other approaches^ certain terms need 
definition. The term classification is used here to refer to the categorization 
of the courses in which students enrolled over the duration of their baccalaure- 




ate program. It is the systeanatic and unique way a college or university labels 
and arranges its courses (i.e.. Honors 101, French 340, etc.); that 8chen» or 
arrangement of classes is already known in a disaggregate form on student tran- 
scripts. Identification is the allocation of individual courses to be 
established in categories on the basis of specific criteria (i.e.. Biology 205 
is classified by many universities as a st^hoioore level class in the department 
of Biology). 

Discriminant analysis is a process undertaken to differentiate between 
groups formed on an a priori basis (See Biglan, 1973a for an example), it is 
the objective of discriminant analysis not to discover groups, but rather to 
identify a set of characteristics that can significantly differentiate between 
the groups. The process allows the analyst to allocate new cases to one of the 
a priori groups with the least amount of error. In contrast, cluster analysis 
recovers groups representing particular patterns from diverse populations (Lorr, 
1983; Romesburg, 1984). In the model developed to analyze coursework patterns, 
civ er analysis is used to classify courses according to student achievement 
criteria, while discriminant analysis is used to test and provide secondary 
validation of the cluster groupings and to identify those criteria which 
significantly differentiate one cluster of coursework from another. 

Factor analysis is different from cluster analysis in that its attention is 
on the similarity of the variables (attributes). The aim is to identify a small 
number of dimensions (factors) that can account for individual differences on 
the various measures or attributes. Thus, the aim of factor analysis is to 
reduce or consolidate the number of attributes of a variable set while the pur- 
pose of a cluster analysis is simply to classify or taxonomize data into groups 
on the basis of a set of attributes. Miller (1969) examined 48 common nouns; 
through cluster analysis he identified five subgroups referring to living 
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things, nonliving things, quantitative terns, social interactions, and 
emotion . Another exaisple of cluster analysis is Paykel*s (1971) analysis of 
165 depressed patients. Uring syn^ptom ratings and historical variables, he 
grouped the patients into four cl\isters: the retaurd psychotic, the anxious, the 
hostile, and the young depressive. Cluster analysis refers to a wide variety of 
techniques used to classify entities into hcoiogenous subgroups on the basis of 
their similarities. 

The end products of cluster analysis are clusters or pattern sets. Since 
the exact nuntoer and nature of the coiirse patterns is not lcnc»m in advance, the 
clustering process is actually technically preclassif icatory. in other words, 
cluster analysis techniques are used to construct a classification scheme for 
unclassified data sets. In this way, cluster analysis ea?>irically arranges the 
courses of a college curriculum using student decision-making behavior (as rep- 
resented on transcripts) as the priinary source of information. The courses are 
classified in a hierarchical dendrogram or tree. The relationship between 
courses is determined by their similarity on the criteria used in the classifi- 
cation. In this way, the similarity between courses is determined empirically, 
rather than by arbitrary concepts (i.e., "life sciences") or levels (i.e., 
"freshmen level survey"). This conceptual/ ea?>irical approach was selected due 
to the lack of agreement in the higher education literature on a coomran research 
paradigm, model or philosophy for the organization of coursework (Bergquist et 
al., 1981; Biglan, 1973a; Furhmann and Grasha, 1983; Gaff, 1983; Rudolph, 1977; 
Sloan, 1971; Veysey, 1973). 

Cluster analysis conforms to the conceptual restrictions placed on the mod- 
el in order to assess the effect of coursework patterns on student learning. 
Cluster analysis provides a statistical procedure for examining coursework using 
multiple criteria. It can accommodate both quantitative and qualitative attri- 
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butes of varying dimensions. Thus, the criterion selected need not be test 
scores; ncminal, order, interval and ratio data have been successfully used as 
attributes in cluster analysis (Romesburg, 1984). Cluster analysis uses these 
attributes to arrive' at patterns of coursework independent of any institution- 
ally prescribed a priori distinctions among courses. It therefore is capable of 
testing notions of cooibinations, sequence and prcygression of covirses within the 
college curriculum- It leads to the discovery of clusters {or patterns) of 
coursework in student transcripts, based on the attributes of students* general 
learned abilities. Since the purpose of the cluster analytic model is to group 
coursework homogeneously according to its relation to student learning outcomes 
(Lorr, 1983; Roroesburg, 1984), cluster analysis was chosen as the primary 
methodology for analyzing student transcripts in this Differential Coursework 
Patterns Project. 



General procedural steps 

This section describes the steps in the statistical process anbodied in the 
cluster analytic model. These steps address the five research objectives 
(discussed in Section 1) of the model. Student residual scores are derived. 
Student transcripts are examined, aiid courses reported on them are clustered 
into patterns based on the score gains of the students who enrolled in the 
courses- Resulting patterns of coursework are again analyzed and classified 
according to the sequences and combinations of courses within the cluster and 
according to term of enrollment data found on students transcripts- Also, 
resulting patterns of coursework are analyzed and classified according to 
attributes associated with the educational environment of the college or 
university: (a) the type of college or university, as indicated by Carnegie 
classification (1987), (b) the type of general education degree requirements of 
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the institution, as indicated in the college catalog or bulletin, (c> the type 
of academic discipline or field of study, as indicated by the course prefix on 
the transcript, and d) the student demographic characteristics, as indicated on 
the d«nographic questionnaire completed by the student at the tiaie of GRE 
testing. Hypothesized patterns of coursework generated from one set of student 
transcripts may be validated through the raplicaticn of the cluster analytic 
nwdel using a second sample of student treinscripts . 

Quantitative versus qualitative measures 

As previously described in Section 1, there is nrore than one view of what 
constitutes representation of a college or university curriculum within a sanple 
of student transcripts. One view suggests that only those courses in which 
students most frequently enroll constitutes the curriculum associated with gen- 
eral learning in the undergraduate program. A second view holds that any course 
offered may contribute to the general learning of students. The first view im- 
plies a more restricted view of the curriculum than does the second. These con- 
trasting views resulted in the development of two alternate procedures for as- 
sessing the associated effects of coursework patterns on general learned abili- 
ties. Reported in the following section are the results of the first procedure, 

quantitative cluster analysis. The second procedure, qualitative cluster 
analysis , is descril>ed thereafter. 

The cluster analytic model uses multiple measures of general learned abili- 
ties as attributes with which to classify courses taken into patterns. These 
attributes can be expressed quantitatively or qualitatively. For example, a 
sophomore level mathematics class (i.e.. Math 201) can be described according to 
the mean residual score of students (from the sample) who enrolled in the 
course. Math 201 can also be described nominally; here the researcher singly 
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notes irfhether one oi niore students with high residual scores enrolled in the 
course. Bo^Ji U.c ^iuantitative and qualitative descriptions of Math 201 serve to 
determine the relation of the course to other courses according to the item-type 
criteria variables. 

When a sang>le of students is used to examine the effects of a particular 
college curriculum on general learning^ there are a limited nuntoer of courses 
within the curriculum which can be analyzed quantitatively. The GSU Historical 
Group exan?>le^ previously described^ illustrated this problem. Only a limited 
percent of all courses appearing on the sanf>le transcripts can be analyzed if 
the number of students enrolling in a given course is a concern in the 
analysis. However, such quantitative analysis of the curriculum can yield much 
more accurate information regarding the effect a particular course may have on a 
given measure of student general learned ability. To generalize about a course 
on the basis of 5 or more student residual scores provides a level of 
information that far exceeds that of siioply noting whether any student who 
performed well on a given measure enrolled in that course. 

There are advantages and disadvantages to either the quantitative or tho 
qualitative approach* In the quantitative analysis, a limited number of courses 
can be examined, but, in practice, those courses are those in which most stu- 
dents enroll and enconqpass all those in which students are required to enroll. 
Math 101, a required mathematics course in a college's curricul\im, would be 
included in those courses examined in a quantitative cluster analysis since all 
students are required to enroll, while Math 450 designed primarily for senior 
level math majors would not be included — assuming the sample of students is 
random and not confined to mathematics students. 

There are those, however, who may argue that it is the advanced coursework 
within a given discipline which facilitates general student learning. It has 
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been suggeste-^ that the study of liberal arts disciplines teaches students a 
mode of inquiry which facilitates their learning of other forms of knowledge, 
abilities and skills (Biglan, 1973a, 1973b). Similarly, courses with tradition- 
ally restricted enrollments may not appear in an analysis of coursework selected 
by the frequency of enrollment. Analysis of the effect of credit for study 
abroad or honors programs or the assessront of coursetrork patterns of specific 
groups of students might not be possible. Therefore, under these and related 
circumstances, it is desirable also to exnmine as many courses of a student's 
transcript as possible, rather than restricting the analysis to only those 
courses in which students most frequently enroll. 

Examination of all courses on a student's transcript may not be feasible. 
Some courses may have only one student enrolled frcan the sanple group, the 
cohort, or population of students examined. Recall that in the cluster analytic 
model, a student's GRE item-type residuals are attributed to all the courses in 
which he/she enrolled. The contribution of individual courses to the curriculum 
is calculated as the sum of the effects of the students who enrolled in those 
courses. Courses with low enrollments from the group being examined have higher 
margins of error because the effects are discerned from a smaller number of 
students. Thus, courses with an enrollment of one student frcm the saiqsle group 
do not provide a basis for quantitative analysis, while courses with limited 
enrollment (2 or more) may amenable to the treatment of that enrollment 
solely as a nominal variable. 

In a quantitative cluster analysis, the metrics used for each course are 
the mean GRE item-type residuals which contain interval information about the 
gains of students who enrolled in the course. In a qualitative cluster 
analysis, the metrics used are whether students with high residual scores did or 
did not enroll; the metric is reduced to a dichotoroous nominal variable. There 
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is a trade-off in a qualitative cluster analysis between inclusiveneas of the 
curriculuffl and precision of the information. 

Any quantitative attribute, such as a GRE itero-type residual, can be di- 
chotomized and converted into a binary attribute (Soiderberg, 1973). Such a pro- 
cedure lessens the precision of information in the data set because the process 
is irreversible. The data from an interval scale is collapsed into a no«inal 
one. It is coraKsnly held that ratio scales provide more precise information 
than interval scales, that interval scales are more precise than ordinal ones, 
and that all the preceding are more informative than nominal scales. H(»«ever, 
the choice of scales is constrained by different factors. 

First, institutional researchers are often under monetary constraints. The 
costs of obtaining test scores for all college graduates, for exan^jle, may not 
be feasible on an on-going basis. Hence, it may not be practical to gather the 
number of student transcripts and assessment information needed to use the quan- 
titative cluster analysis with courses other than those in which students roost 
frequently enroll. 

Second, institutional researchers have a choice between an intensively de- 
tailed picture of the curriculum using the ratio data of mean residuals or a 
less detailed picture provided by binary information. As has been previously 
discussed, there are occasions when the scope of the analysis is to be preferred 
over the precision of the analysis. 

Third, "data do not automatically inform the researcher" (Rcmesburg, 
1984). To have meaning, transcript and test data must be interpretable within 
a curricular context. The primary question is, "vmich coursework patterns 
contribute to general student learning?" The secondary questions are, "How much 
do the patterns contribute?" and "What is their relative contribution?" 
Qualitative analyses are not categorically inferior. In this case, a 
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qualitative aiialytic question precedes the one which say be answered 
quantitatively . 

Procedure 1; Quantitative cluster analysis 

Described below are steps required in Procedure 1 (Quantitative Analysis) 
to assess the effects associated with the coursework patterns on the general 
learned abilities of college students. The research design uses as data sources 
transcripts and GR£ and SAT test scores frc^ a sanple of students. The 9 item- 
type categories of the General Tests of the Graduate Record Examination are used 
as measures of general learned abilities of college seniors. These seniors' SAT 
scores are used as variables to control for the acadeanic abilities of these 
students when they first entered college. The student transcripts are used as 
the record of the sequence of courses in which these seniors enrolled. 

The first objective of the cluster analytic model is to determine the stu- 
dent iji5>roveinent in general learned abilities over the time of their 
baccalaureate program. To do this, first the residual score of each item-type 
for each student is calculated; the residual score is the difference between the 
student's actual score and the score predicted by the student's corresponding 
SAT score. Thus, for each student outctsne measure there is a student residual 
score for each person in the san5>le group. 

The second objective is to determine patterns of coursework on the student 
transcripts which are associated with student residual scores. This is 
acccmiplished through cluster analysis, using student residual scores (GRE item- 
type residuals) as attributes of the courses in which student., ,.^olled. 

A raw data matrix consisting of columns of courses and ows of residual 
scores is created. The mean residual score for all the students in th^ « ^le 
who enrolled in a given course is calculated and becoBeg the metric value for 



^hat course* The correlation coefficient is used as the reseablance i»ef f Iciwit 
to transform the data matrix into a resemblance matrix, wherein the similarity 
of residual scores for students enrolling in one course can be compared with 
those enrolled in another course* Once the reseriolaiKre natrix indicating the 
proportional relationship of courses is established, a clustering method ic 
selected and executed to arrange a tree or dendrogram of ccnirses related by the 
man residual of each course. Next, a discriminant analysis is perfonmd on the 
resulting clusters of coursework to (a) determine the extent to which the 
courses have been correctly classified according to the 9 n^an student residual 
scores, (b) to determine which of the 9 mean residual scores were correlated 
with particular discriminant functions, and (c) to determine which coursework 
clusters exhibited high mean residual scores relative to each discriminant 
function. From the discriminant analysis an association can be inferred between 
coursework patterns (clusters) and general learned abilities {mean residual 
scores on 9 criterion variables). The cluster-analytic procedure groups courses 
frequently chosen by students according to the strength of their associated 
effect on the student residual scores. 

Described in greater detail below the steps followed in this cluster 
analytic procedure: 

Step 1 , Calculate a student residual score for each item- type (attribute) of 
each student GRE. This step removes the predictive effect of the 
student's SAT scores from the GRE item-type, thereby controlling for the 
academic ability of the student upon entrance to college. For GRE 
Quantitative item-types, the effect of the student's SAT Math score is 
partialled out. For the GRE Verbal item-types, the effects of the SAT 
Verbal score is partialled out- For the GRE Analytic item-types, the 
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effect of the cootoinea SAT Verbal and SAT Math scores are partlalled 
out. In this way^ the student's acadesdc abilities prior to entering 
college is controlled when calculating student residual scores. 



Step 2 . Calculate the iroan residual score for each course enrolling 5 or ©ore 
students fron the sample group. Cross-listed courses are standardized 
so that they have only one identifier. Cross-listed courses include 
those with identical nunibers that have different labels. Courses with 
the same course identifier but with abstentious ly different content 
(i.e.^ "Music 101: Voice" and "Music 101: Piano") are excluded frcan the 
analysis; however catalog changes are accounted for. If Math 201 in 
1982 was renumbered as Math 211 in 1985, Math 201 and Math 211 for those 
years are treated as the same course for the purposes of analysis. 



The proportion of courses included in the analysis is related to a) the 
extensiveness of the course listings in the curriculum^ and b) the size 
oi the student sample- The more extensive the curriculum, the less fre- 
quently 5 or more students from the sanple will have enrolled in the 
same course. Likewise, the smaller the size of the student sairple, the 
less frequently 5 or more students from the saii^le will have enrolled in 
the san^ course. 



Step 3 . Create a raw data matrix by using the mean residual scores of the san^^le 



and the courses found on 5 or more of the student transcripts. The rows 
in the data matrix consist of the 9 GRE item-type scores while the 
columns represent those courses enrolling 5 or more students. Each cell 
value of the matrix is a mean GRE item-type residual score for those 
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group students enrolling In a specific course. For example, the course 
(object) in the first column in the data matrix is ANTHROPOLOGY 101, and 
the student outccffl» measure in the first row of the data matrix is DATA 
INTERPRETATION. The student residual scores are .40, .45, .50, .55, and 
.60; the mean residual score, therefore, is .50 and is entered as the 
metric variable in cell (1*1) of the matrix, since the variables in 
each row are of the same magnitude, and therefore, have coirparable 
effect on the resulting cluster analysis, the data matrix does not need 
to be standardized (Raiwsburg, 1984). The cluster analysis will 
taxoncmiize courses in the curriculum according to whether students who 
showed positive gains on each item-type were enrolled in the courses. 
This step prepares a raw data matrix to be used in a general cluster 
analysis based on quantitative data. 



Step 4 . Select a resemblance coefficient. The resemblance coefficient 

(Rcmiesburg, 1984) is also called the similarity index (Lorr, 1983). The 
purpose of the resemblance coefficient is to explain the similarity (or 
dissimilarity) of each cell to each of the other cells in the data mat- 
rix; it is expressed mathematically. There are many resemblance coef- 
ficients; each will express the similarity between courses (objects) in 
a slightly differently way. Each coefficient is appropriate for achiev- 
ing slightly different research goals. 

The resemblance coefficient selected for this study is Pearson's 
product -moment correlation coefficient- It is appropriate for use with 
ratio data. The resemblance coefficient will indicate the similarity of 
courses to each other according to the 9 item-type residuals 
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(attributes) coded in the data matrix. The r8s«i4>lance coefficient 
e3q>resses the relationship of two courses proportionally. 



St^P^. Calculate a resoitolance matrix fron the raw data matrix. The 

resemblance matrix is calculated by transforming the raw data matrix 
using the correlation resonblance coefficient, in this cluster analytic 
model, the data matrix consists of quantitative data described by 9 
attributes ranging in value from 1.00 to -1.00. In the resemblance 
matrix, the columns represent the first course (object) in a pair, the 
rows represent the second course (object) in a pair. The resemblance 
coefficient (Pearson's r) is entered into each cell. The cell value 
represents the extent to which the attributes on the first course 
explain the variance in attributes on the second course. The 
resemblance coefficient serves as a measure of similarity between one 
course and each other course in the calculation of clusters or 
coursework patterns. 

SteE_6. Select and execute the clustering method. A resemblance matrix is 
transformed into a tree of related courses (objects) by use of a 
clustering method which is a series of steps that removes values from 
the resemblance matrix. Therefore the size of the matrix is reduced. 
Each time a value is removed f ran the resemblance matrix it is placed in 
the cluster tree or dendrogram. In the last step, the resemblance 
matrix disappears completely and the tree is cOT^leted as the last value 
is inserted. 
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Romesburg (1984, p. 139) recOTBwnds the unweighted pair-group method 
using arithmetic averages (UPGMA), also known as the average linkage 
method. UPOiA is recommended over single linkage clustering n»thod 
(SLINK) and conplete linkage clustering method (CLIMK) for two reasons. 
First, it can be used with any resemblance coefficient, while SLINK and 
CLINK are designed to be used with interval and ratio data in a 
quantitative data matrix. Second, it judges the similarity between 
pairs of clusters in a less extreme manner than do SLINK and CLINK. The 
average linkage method (UPGMA) is available on SPSSx, SAS and BIIDP 
statistical packages. 

Steg_2. Determine the optimum number of coursework clusters. Cluster analysis 

is a procedure for taxonomizing or classifying coursework data. The num- 
ber of groups or patterns in which the data is classified according to 
the criterion variables is an arbitrary one. Once relationships between 
coursf»s have been determined, the researcher must decide on how many 
groups in which to put the data. Discriminant analysis provides a means 
to test the secondary validity of the coursework pattern groupings. 

By computing successive cluster analyses for different numbers of clus- 
ters and then conducting discriminant analyses on the resultant group- 
ings, one can identify the number of clusters which has the highest pre- 
dictive value, given the criterion variables used. Using the DISCRIMIN- 
ANT program in SPSSx, for example, will identify how many members of 
each coursework pattern or cluster were correctly classified, how many 
could be classified in other patterns, and what was the overall percen- 
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tage of correct classification. 



The number of clusters with the highest predictive value may not be the 
sole objective in examining the merits of different cluster solutions to 
the cluster analysis. Theoretically, a four cluster solution may have 
high predictive value for GRE itean-types because the item-type residuals 
are forced into three discriminant functions which should approximate 
the GRE sub-scores. Likewise, a 10 cluster solution may prove to be 
slightly less predictive, but the 9 GRE item-type residuals may be more 
clearly associated with discrete coursework patterns. Careful visual 
inspection of the cluster dendrogram often suggests appropriate cluster 
solutions to test using discriminant analysis. 

Step__8. Determine which criterion variables contribute significantly to which 

discriminant functions. DISCRIMINANT in SPSSx, for exao?>le, calcv'^ates 
the pooled within-groups correlations between the discriminating vari- 
ables (in this case. Me mean residual scores on the 9 item-types) an^ , 
the canonical discriminate functions. Large positive ar.d negative cor- 
relations are identified. Eigenvalues for each discriminant function 
are assessed- Eigenvalues express the proportion of variance in student 
scores explained by the discriminant function. Discriminant functions 
that explain less than 5 percent of residual score variance or that have 
a probability of error exceeding .001 are discarded. Next, the group 
means for each coursework cluster can be examined, in this manner, the 
patterns of coursework associated with one or more mean item-type 
residual scores can be identified. 
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step 9. Repeat Steps 1 to 8 using a second cohort of students. Following Steps 
1 through 8 will produce a set of hypothesized relationships between 
coursework patterns and student residual scores on 9 criterion measures 
of general learned abilities. Hypothesized relationships cannot be 
tested or validated using the same data. Therefore, a second 
institutional san5>le is dra%ni. A second group of students are tested 
and a second set of transcripts and student residual scores are 
evaluated. Repeated use of the model should refine and clarify members 
within each coursework pattern. 



Through the above 9 steps, the cluster analytic model classifies the most 
frequently enrolled courses according to their associated effect on student 
residual scores. Procedure 1 classifies courses according to a ratio index of 
similarity to other courses. While Procedure 1 may examine only a fraction of 
all the courses in a college curriculum, it does provide a means to 
differentiate the effect of required courses or courses in which most students 
enroll. For example, in the GSU historical database used in model -building and 
testing, a five percent san?>le of student transcripts enabled an examination of 
only five percent of courses appearing on those transcripts (the percentage of 
courses enrolling 5 or more students fran the saii?)le group). However, the 
courses examined in that 5 percent corresponded closely to those courses 
identified as meeting the College's distributional degree requirements in 
general education. Similarly, 146 Ithaca transcripts (Sample #1) yielded 405 
individual courses enrolling 5 or more of the students from a list of 1,136 
unduplicated courses. While these 405 courses represented over one-third of all 
courses on the transcripts, a majority of the courses were those listed in the 
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Ithaca catalog as n^eting the general education requireiMnts of the College 




' Pescription of the .:;e orqia State Native Coanbined Sampl, 



This section describes the Georgia state coo4>ined sample (1986-87,1987-88) 
group of native graduating seniors. 

Eligibi lity Requirefflents 

Two criteria were established to determine eligible students from Georgia 
State (Natives) to participate in this study. Students had to meet both 
eligibility requirements in order to participate in the study: 

^' « graduation date of May 1987 or an expected graduation date of 
December 1986 or May 1987 (these criteria are for inclusion in 
sanple #1, the same criteria, except one year later, are used for 
saii5>le #2); and 

2. at least 84 total accumulated credits. 

Any students with fewer than 84 total accumulated credits were excluded 
since, despite their "expected graduation dates" listed in the College's 
database, they would not be able to graduate by May 1987 (or May 1988 for second 
sample) taking normal course loads. 

Combined Samples #1 and #2 

The Georgia State Native Group consisted of the transcripts and test scores 
for 168 students from the combined sample #1 and san^le #2. While the Georgia 
State Native Group did approximate the characteristics of graduating seniors for 
that year, several minor variations between the combined san^sles' student 
characteristics and those of the population as a whole are reported below. 

Gender has been shown to be a significant factor in the academic 
performance of college undergraduates. Over one-half (56.5%) of the sample were 
female, while 43.5 percent of the population were female (see Figure 3-1). 
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Race and ethnicity also have been shown to be Strang predictors of acadeoiic 
performance. Sixty-nine percent of the coadbined Group were white ^ conpared to 
seventy-four percent of the graduating population « Blacks were overrepresented 
in the Group by seven percent while Asian students were underrepresented by six 
percent (see Figure 3-2). 

Major field of study has been shown to be correlated to performance in the 
GRE examinations. The distribiition of majors in the condbined Group approximated 
that of the population • Majors in Information Systems and Journalism were 
slightly overrepresented while Finance, Studio^ Accounting, and Marketing were 
slightly underrepresented (see Figure 3-3), 

Figures 3'4a and 3-4b present the SAT scores for the ccsribined Group and the 
population. The confcined Group scored better on the Math portion (482.3) of the 
exam than the Verbal portion (471.3), as did the population, but the san?)le 
scored higher on the verbal. 

Figure 3-1. Distribution of Georgia State Native Combined San^le: Gender 

Combined Sample Population 
Gender N PERCENT N PERCENT 

Female 95 56.5% 1173 

Male 73 43.5% 9'i7 

TOTALS 168 100.0% 2110 



55.6% 
44.4% 

100.0% 




7 



r: 
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Figure 3-2. Distribution of Georgia State Hative Group: Ethnicity 



Ethnicity 



Native GrtHip 
N PERCENT 



Pcgmlation 
N PERCENT 



Not spfecified 
Black 

Native American 

Asian/Asian American 

Hispanic 

White 

Foreign 

Other 



8 
38 

0 

3 

3 
116 

0 

0 



4.76% 
22.62% 
.00% 
1.79% 
1.79% 
69.05% 
.00% 
.00% 



336 



160 



47 



.00% 



15.92% 



.00% 



7.58% 



2.23% 



1,563 74.08% 



.00% 



.19% 



TO'.'ALS 



168 



100.00% 



2110 



100.00% 
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Figure 3-3. Distribution of Native Group: First Major 

Native Group Population 
^^^or N PEROan? N Percent 
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English 
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Mathematics 

Music 

Nursing 

Office Administration 

Orchestra 
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roiiuicai Science 
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3.0% 


93 


4.4% 


Physical Theraphy 


1 


.6% 


15 


.7% 


Real Estate 


0 


.0% 


13 


.6% 


Real Estate & Urban Affairs 


0 


.0% 


10 


.5% 


Risk Management 


0 


.0% 


21 


1.0% 


Respiratory Theraphy 


0 


.0% 


16 


.8% 


Secondary Education 


0 


.0% 


38 


1.8% 


Sociology 


1 


.6% 


20 


.9% 


spanisn 


2 
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Figure 3-4a. Sumnary of SAT Scores and Grades for Native Group 



C3 s szsESsa Bssasscaaas 



standard standard 
Score M Mean Deviation Range Error 



Verbal 
Math 

SAT Total 
Cumulative GPA 



168 
168 
168 
168 



471.27 
482.28 
953.55 
2.80 



100.93 
99.23 
172.46 
.67 



270-710 
260-750 
550-1410 
0-4.00 



7.79 
7.66 
13.30 



SAT Verbal Scores 
Midpoints 



N 



Percent 



Distribution 



300 


12 


7.5% 


****** 


350 


21 


13.2% 


*********** 


400 


23 


14.5% 




450 


33 


20.8% 


****************** 


500 


29 


18.2% 


**************** 


550 


19 


11.9% 




600 


22 
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************ 


650 


4 
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** 


700 
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*** 
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.5% 


««**** 


400 


18 


11, 


.3% 


********** 


450 


38 


23, 


.9% 




500 


35 


22. 
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******************* 
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************** 
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Figure 3-4b. Sunaary of SAT Scores and CPA for Hatlve Groitp & Population 

SAT Part Native Population 

Score N=168 N=2110 



Verbal Mean 471.27 440.46 

Math Mean 482.28 475.39 

Cumulative GPA 2.80 2.556 



Figure 3-5. Entering Ses^ster of Native Group 



Entering Semester 




N 


PERCENT 


1970 




2 


1.2% 


1971 




1 


.6% 


1972 




4 


2.4% 


1973 




3 


1.8% 


1974 




2 


1.2% 


1975 




4 


2.4% 


1976 




iO 


6.0% 


1977 




5 


3.0% 


1978 




5 


3.0% 


1979 




15 


8.9% 


1980 




20 


11.9% 


1981 




21 


12.5% 


1982 




26 


15.5% 


1983 




26 


15.5% 


1984 




23 


13.7% 


1985 




1 


.6% 


TOTALS 




168 


100.0% 
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Figure 3-6. Planned Year of Graduation: Native Group 



Planned Year 
of Graduation 



N 



PERCENT 



Ho Response 

1966 

1974 

1978 

1979 

1981 

1982 

1983 

1984 

1985 

1986 

1987 

1988 

1989 

1990 

TOTALS 



13 
1 
1 
1 
3 
1 
1 
7 
5 
4 
19 
43 
57 
10 
2 

168 



7.7% 
.6% 
.6% 

.6% 
1.8% 
.6% 
.6% 
4.2% 
3.0% 
2.4% 
11.3% 
25.6% 
33.9% 
6.0% 
1.2% 

100.0% 



Figure 3-5 shows that the majority of the Native Group entered the 
institution in the fall 1982, fall 1983, or the fall 1984 terms. The majority 
of students in the Native Group reported graduation dates for a bachelor's 
degree either during the 1986-87 or the 1987-88 academic year (Figure 3-6). Six 
percent of the students planned to graduate in the 1988-89 academic year. 

Students in the ccanbined sample were clearly planning some form of 
post-baccalaureate study (Figure 3-7). Two-thirds (66.1%) planned to pursue a 
Tester's degree, while nearly 17 percent planned to enter a ik>ctoral program. 
Only three percent had no plans for subsequent graduate study. 

The educational attainment of parents has been shown to be positively 
correlated to student achievement in college. Nearly 31 percent of the mothers 
and 14.3 percent of the fathers of students had attained a high school diploma. 
Approximately one- tenth of both fathers and nwthers had attained at least the 
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bachelor's degree (Figure 3-8). 

Figure 3-7. Degree Objectives for Georgia State Native Group 

Degree Objectives N PERCENT 

No Response 21 12.50% 

Non-degree Study 5 2 98% 

Master's Degree 111 66.07% 

Intermediate Degree (e.g.. Specialist) 3 1.79% 

Doctorate (Ph.D., Ed.D.) 28 16.67% 

Postdoctoral Study 0 .00% 

168 100.00% 
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Figiire 3-8. Educational Attaimiaant of Parents for Georgia State Native 
Group 



Hlqheat Level of Father Mother 

Education Cc»?>leted N Percent N Percent 
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22 


13 


.10% 


12 
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.14% 
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14 
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52 


30, 


.95% 


Business or Trade School 


25 


14 


.88% 


15 


8, 


.93% 


Sc^ne College 


18 


10 


.71% 


12 


7. 


.14% 


Associate Degree 


10 


5 


.95% 


7 


4. 


,17% 


Bachelor *s Degree 


17 


10 


.12% 


16 


9, 


.52% 


Some Graduate/Professional School 


6 


3 


.57% 


1 




.60% 


Graduate/Professional Degree 


13 


7 


.74% 


10 


5. 


.95% 


TOTALS 


168 


100 


.00% 


168 


100. 


.00% 















Figure 3-9. Extent of Cr«ranunity Service Activities for Native Group 

Hours/Week in Conrounity Service n PERCENT 



Ko response 
0 hours 

I- 5 hours 
6-10 hours 

II- 20 hours 

More than 20 hours 

TOTALS 



19 11.31% 

61 36.31% 

64 38.10% 

16 9.52% 

2 1 . 19% 

6 3.57% 

168 100.00% 
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Figure 3-10. Important Honors and Awards for Native Group 



Type of Honor/Award 



Percent 



10.71% 
12.50% 
7.14% 

7.14% 



No response 

Student government or orcjanization 21 

Professional (an award or prize for field work or 12 
publication of a scholarly article or book) 

Caiawnity service (election or ai^intaient to a 12 
community service unit, activity, or group) 

Literary (editing the college paper, yearbook, or 18 10.71% 

literary magazine or having a poem, story, or 
article published in a public paper or magazine) 

Artistic (a high rating in a music contest, a part 13 
in a play, opera, or show, or an award in an 
art coanpetition) 



7.74% 



Athletics (a letter in athletics) 



6 3.57% 



None of the above categories 
TOTALS 

168 100.00% 



Over one-half (52.4%) of the combined saii?,le students had performed sc^ 
community service during the past year, but for 38.1 percent of these students 
this had conprised less than five hours per week (see Figure 3-9). 

Over one-half (59.5%) of the sair?5le students \ad earned some form of 
professional, coomiunity service, literary, artistic, athletics, or student 
government honor, or award. Prior research had shown such distinctions to be 
highly correlated to student performance, persistence, progress, and degree 
attainment in college. 
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Figure 3-11. Suoaiary of Georgia State Native Group 



Sai^le Size: 
Gender: 
Bace: 

Major Area: 

College: 



168 students 

95 females (56.5%) and 73 males (43.5%) 

116 out of 168 are white (69.05%) 
38 students are black (22.62%) 

8 or more students majored in each of Accounting, Journalism^ 
Marketing, Psychology, Information Systems, Criminal Justice, 
and Management 

7 students (4.2%) in Allied Health 
69 students (41.1%) in Ar*:s & Sciences 
58 students (34.5^) in Business Administration 
11 students (6.5%) in Educati'^n 
23 students (13.3%) in Urban Affairs 



Figure 3-11 summarizes the characteristics of Georgia State Native Groip. 
The majority of students were majoring in programs within the colleges of 
Business Administration and Arts and Sciences. 
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TV. Peterroining Student Learned Abilities 

GRE residucl scores 

To control for the effects of the incosning ability of students, the 

predictive effect of SAT scores were partialled fxxan GRE item-type scores. For 

this^ 9 GRE item-type residual scores were developed as follows: 

GRE Verbal item-type residuals; 

ANA: Analogies 18 questions 

SC: Sentence Completion 14 questions 

RD: Reading Cosrprehension 22 questions 

ANT: Antonyms 22 questions 

GKE Quantitative item- type residuals; 

QC: Quantitative Cc^ny^^urison 30 questions 

RM: Regular Mathematics 17 questions 

DI: Data Interpretation 10 questions 

GRE Analytical item-tyj^ residuals; 

ARE: Analytical Reasoning 38 questions 

LR: Logical Reasoning 12 questions 

Each of the 4 GRE Verbal item-type scores were regressed on the SAT Verbal 

scores. Each of the 3 GRE quantitative item-type scores were regressed on the 

SAT mathematics scores. Each of the 2 GRE analytical item- type scores were 

regressed on the SAT total scores. These GRE item-type residual scores were 

referred to as student residual scores^ that is^ the improvement students showed 

in general learned abilities frOTi the time they entered college to the time of 

GRE testing during their senior year. 

Relif^ility and correlation of GRE item-types 

Prior to partialling the effects of the stuf?ents' SAT scores from their GRE 
iter x.rpe scores^ the reliability of the GRE item- types for this sample was 
tested. Next, the correlation between the GRE item-types and the SAT sub-scores 
and total score was examined. Finally, a regression of GRE item- types on SAT 
sub'scores was conducted to calculate student residual scores for each GRE 
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item-type. 

A preliminary question in the analysis of GRE item-types and sub-tests is 
their reliability within the sample group. Three factors typically contribute to 
the reliability or unreliability of test scores (Ebel, 1972). The first factor 
is the appropriateness and deJinitiveness of the questions. On one hand, the 
appropriateness of the questions is presuned by the widespread acceptance of the 
GRE as an examination used in graduate school admissions. On the other hand, 
the appropriateness of the items and item-types may be questioned relative to 
the goals of the general education c rriculum of the institution. In this 
sense, the reliability of the GRE may vary frcan institution to institution. 

A second factor contributing to the reliability of test scores is the 
consistency and objectivity of the person (or in this case, machine) who scores 
the examinations. All the test responses are read by an optimal scanner and 
scored by a congjuter at Educational Testing Service, The accuracy of this 
equipment relative to the task was presun»d and not tested. 

A third factor contributing to the reliability is the constancy or 
stability of a student's ability to perform the tasks presented in the test. 
Students may vary from hour to hour or from day to day in their alertness, 
energy and recall; these may affect test performance, reducing the reliability 
of the scores. According to procedures established by the Educational Testing 
Service for the administration of the Graduate Record Examination, all students 
were tested on the first Saturday morning (8:00 a.m. to 12:00 p.iA.) in February, 
of each test year. 

Reliability is not merely the property of the GRE itself but rather of the 
individual item-types relative to the student grcup examined. The more 
appropriate the test is to the group of students, the higher the reliability of 
the scores. Ideally, the reliablity of a set of scores may be determined using 
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the correlation coefficient between that set of scores and another set fron an 
equivalent test of the members of the same group, in many testing situations, 
inc ing th ones described in this report, a test-retest method of determining 
the reliability of GRE item- types was not available . 

The Guttman Split-Half method of determining reliability estimates 
reliability by splitting the sanple into halves and determining the correlation 
between the scores in the two groups. The results of the split -half method are 
dependent upon the manner in which the group is halved. Cronbach's alpha is a 
statistic designed to overcorae this problean. it ib a generalized formula 
representing the average correlation obtained from all possible split-half 
reliability estimates. 

The results of the reliability analysis for Georgia State Native Sanple #1 
is presented in Figure 4-la; the reliability analysis for Georgia State Native 
Sample #2 is displayed in Figure 4-lb. Since different toxos of the GRE were 
used each year, the reliability results are presented for each individual 
sample. For the purposes of this study, reliability coefficients at or above a= 
.65 were deemed satisfactory (Mehrens & Lehmann, 1969). Due to the exploratory 
nature of this research, lower reliability coefficients were accepted. In Sample 
Group #1, Analogies (a= .60), Data Interpretation (a = .59), and Logical 
Reasoning (a = .52), evidenced low reliability. In Saagjle Group #2, Sentence 
Con^Jletion (a= .64), Analogies (a= .53), Data Interpretation (a= .47), and 
Logical Reasoning (a= .58) showed low reliability. In both san^sles, the 
reliability of the individual item-types tended to increase with the number of 
items congsrising the given item-type. In the cluster analytic model, the SAT 
sub-scores are used as measures of entering student ability. Prior to regressing 
GRE item- type scores on SAT scores, it is iir9>ortant to determine the extent to 
which GRE item- types and SAT sub-scores are correlated. For example, determining 



- 72 - 



SS 



whether the GRE item-type. Analogies, has a stronger correlation with SAT 
Verbal, SAT Math or the total SAT scores will help determine which SAT score 
should be used in the subsequent regression analysis. 



Figure 4 -la. Reliability of Coefficients of GRE Item-Types— Saaf>le 



#1 



GRE Iteai-tvoes 




Number 
of itons 


Cronbach * s 
Alpha 


Guttman^s 
Split-half 


Analogy 


AMA 


18 


.6021 


.5898 




SC 


14 


.6497 


.6976 


Reading Ccoiprehension 


RD 


22 


.7909 


.6878 


Antonyms 


ANT 


22 


.8285 


.8313 


Quantitative Coa?>arison 


QC 


30 


.6714 


.6607 


Regular Matheiratics 


RN 


20 


.6846 


.7172 


Data Interpretation 


DI 


10 


.5855 


.3182 


Analytical Reasoning 


ARE 


38 


.7900 


.6610 


Logical Reasoning 


LR 


12 


.5215 


.4748 


GRE Verbal 


GRE-V 


76 


.9082 


.9250 


GRE Quantitative 


ORE-Q 


60 


.8402 


.7804 


GRE Analytic 


GRE-A 


50 


.8099 


.7375 



^®^i^ili^y of Coefficients of GRE Item-Types— Sample #2 



GRE Item-types 


Code 


Ntimber 
of it«ns 


Cronbach ' s 
Alpha 


Guttman ' s 
Split-half 


Analogy 

Sentence Completion 
Reading Ccsnprehension 
Antonyms 


ANA 
SC 
RD 

ANT 


18 
14 

22 

22 


.5295 
.6425 

.6645 

.7508 


.3401 
.5351 
.4822 
.6242 


Quantitative Comparison 
Regular Mathematics 
Data Interpretation 


QC 

RM 

DI 


30 
20 
10 


.8067 
.6978 
.4732 


.8255 
.6733 
.3017 


Analytical Reasoning 
Logical Reasoning 


ARE 
LR 


38 

12 


.8136 
.5852 


.7919 
.5459 


GRE Verbal 

GRE Quantitative 

GRE Analytic 


GRE-V 
GRE-Q 
GRE-A 


76 
60 
50 


.8550 
.8712 
.8362 


.6549 
.8506 

.8075 



Figure 4-2 indicates strong, positive relationships between C5RE it«n-tjpes 
and SAT scores. For the Georgia State Native Saa^le, GRE Verbal iten>- types were 
strongly correlated to the SM Verbal sub-score with r ranging from .60 to .68. 
GRE Quantitative item-types had strong correlations with the SAT Matheauatics 
sub-score, r raging from .52 to .72. GRE Analytic item-types evidenced strong 
correlations with the SAT Total score (r = .56 and .61) . Results of the 
correlation analysis were comparable to those found in the other institutional 
saii5>les previously analyzed (Ithaca, Mills, Evergreen, and Stanford Sm^le 
Groups). 
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Figure 4-2. Correlation of GRE Itero-Types & SAT Scores—Georgia State Native 
Group 
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.5367 






.7852 


.7860 


.7081 


Mean 




.5306 


.5149 


.6068 



p < -0001 



Intercorrelation of GRE item-types 

The internal validity of GRE item-types can be measured by ccanparing the 
intercorrelation coefficients of GRE item-types. In the Georiga State Native 
Group, the intercorrelations between GRE Quantitative item-types were relatively 
stronger than those between other GRE item-type scores. Each GRE subscore tended 
to have higher correlations with f • GRE item- types constructing the subscore 
than with GRE item- types constructing other test subscores. The analysis of 
correlations among GRE item- types shows that the item-types have strong internal 
validity. 

Wilson (1985) has suggested that GRE (and SAT) item-types may measure 
discrete forms of general education abilities. This assertion served as the 
theoretical underpinning for the use and treatment of GRE item-types as 
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discrete, multiple measures of general learning. To test Wilson^ s assertion, the 
intercorrelation among itOTJ-type scores was further examined (see Figxire 4--3). 

In the Georgia State Native Group, intercorrelations for Verbal item-t]jfj^s 
ranged fran r = .40 (RD/ANA) to r = .62 (AITT/SC). Intercorrelations for 
Quantitative itMi-types ranged from r - .46 (RM/DI) to r = .63 (RM/QC). 
Intercorrelations between Analytic item- types were r « .42 (ARE/LR). However, 
Analytic Reasoning correlated strongly with Quantitative it«n-types ranging from 
.49 (DI) to .61 (QC)« The >ntercorrelational analyses showed that in most 
instances, less than 50 percent of the variance in one itean-type was esq^lained 
by cxi^t of another. 



Figure 4-3. IstercarrelatlaD a£ (S£ ItflD*-TYP^ ^^r Gecryia State Sstii^ 
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' p < .05 p < .001 
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As Figure 4-4 demonstrates/ Georgia State Native students performed well on 
the GRE General Examination. Students answered the questions correctly in 
approximately 100 of the 18& GRE items. Soo^ students attained perfect scores 
on Sentence Con^letion, Antonyms, Data Interpretation ^ and Logical Reasoning 
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itoji- types . 

Hhile GRE raw scores were generally and consistently high among these 
students, differences among scores appeared when the effect of the precollege 
learning (as measured by the SAT) was removed. tAien the theoretical scores (as 
pred. cted by corresponding SAT scores) were cu^^pared with the students' actual 
responses (Figure 4-5), students showed large proportions of change on TOSt 
item-types . 

The greatest amoiint of variance in it«D-t;^ residuals, including the 
greatest standard error and standard deviation, were found in the Analytic 
Reasoning and Quantitative Co3XQ>arisons item- types. The variance in these 
residuals holds iiii>lication for the ensuing cluster analysis in that GRE 
itan^types with greater variance will play a more significant role in sorting 
courses into clusters. As was discovered in the analysis of saiq>le6 frcsn other 
participating institutions, those GRE item-types with smaller variance play less 
of a role in discriminating course clusters. 

As Figure 4-5 demonstrates^ from one-quarter (Data Interpretation) to 
one--half (Quantitative Coicparisons) of GRE item-type score variation among the 
Georgia State Native Group was explained by their SAT scores. All regression 
functions were statistically significant at ,0001. 

Using the student residuals obtained from the regression analysis above, 
the mean residuals for each course enrolling 5 or more students were calculated 
for all the 9 GRE item- types. Such a procedure does not assume that the specific 
gains of the students enrolled in each course were directly caused by that 
course. Rather, the residuals of each student are attributed to all the courses 
in which they enrolled, and the mean residuals for each course serve as a proxy 
measure of student gains. Once courses are clustered by these gains, then 
hypotheses can be generated and tested as to why students who enrolled in a 



given pattern of courses ea^wriencea significant inprovenent on CTie or more of 
the outcomes criteria (i.e., the it«n-type residuals). 



F igure 4-4. The Distrib ition of GRE Scores for Students in the GSU Native Group 

GRE Item-types Nua4>er Minijnuin Maxinum Score Samplb Standard 

of Itans Right Right Range Mean Deviation 
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Figure 4-5. Suaanary of Regression Analysis of GRE Scores~GSU Native Group 

Dependent Variables Georgia State Native Group 

168 Students 
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7. 
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.4596 



p > F = .0001 



^ egression analysis of SAT scores on GRE item-type scores 

To determine the extent to which these students showed iiJ5>rovement over 
their precollege SAT scores the GRE raw scores were regressed on the 
corresponding SAT scores. GRE Verbal item-types were regressed on SAT Verbal 
sub-scores, GRE Quantitative item-types were regressed on SAT Math scores, and 
GRE Analytic item-types were regressed on SAT Total scores. The resulting GRE 
item-rype correlations with corresponding SAT scores were noticeably lower than 
their corresponding GRE sub-scores. This suggests that individual item-types 
also may measure discrete abilities apart from those cf the SAT sub-scores 
and/or they may reflect lower reliability stenniing fro<n che fact that there ar^ 
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fewer iteans coo9)risin9 an Item-type than a sub-score. 

The SAT scores explained sisaller portions of variance in GRE item-type 
scores than in the GRE sub-scores (Verbal, Quantitati\'0 and Analytical—see 
Figure 4-5). The SAT Verbal e3?>lained 42.53 percent of the varia^ice in the 
Sentence Con?>letion item- type among the Georgia State Native Group. The SAT 
Verbal explained 35.75 percent of the variation in the Analogies item-type, 
36.53 percent of thft variation in Reading Ccsnprehension and 46.01 percent in 
Antonyms items among the Georgia State Native Grcmp. The SAT Math scores 
explained 52.06 percent of variation in Quantitative CCT5>arison item responses, 
46.60 percent of variation in Regular Math item-type scores, and 26.86 percent 
of variation in Data Interfretation for the Georgia State Native Group. The 
u^iied SAT Verbal and SAT Math scores (referred to as SAT Total) explained 
37.13 percent of variance in Analytic Reasoning and 30.60 percent of variance in 
Logical Reasoning for the Georgia state Native Group. In all instances, the 
regression nradel proved significant at the .0001 level, suggesting effective 
control measures for the general ' learned abilities of students as they entered 
college as freshmen, 

Georgia State Native Group students entered college with slightly higher 
mean SAT Math score (482.3) than SAT Verbal score (471.3). As these Georgia 
State Native students approached graduation, they remained normatively stronger 
in Quantitative abilities (mean GRE-Q = 476) than in Verbal abilities (mean 
GRE-V = 475). Yet, the regression analysis showed the Georgia state Native 
students evinced large variance in residuals on specific GRE item- types in 
excess of th- ae represented in the sub-scores of the two tests. For Georgia 
State Native students, specific and significant residual variance was 
demonstrated in Quantitative Comparison and Analytic Reasoning. 
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Figure 4-5 coi^«re8 the ejqjlained variance (R-squared) for each GRE 
item-type, raw GRE sub-score and converted GRE sub-score. Only SAT sub-scores 
were available to the research team; these scores are converted scores . The 
actual scores of a student on a particular form of the SAT test is transformed 
relative to teat norms so as to be coo?>arable with other forms of the test and 
with other students tested. A similar process is used with the GRE exams. Raw 
GRE sub-scores for a given form of the test are transformed so as to allow 
coB9>arisons with national norms and with scores on other forms of the test, in 
all cases within the Georgia state Native Group, the SAT accounted for more 
variance in GRE sub-scores than in the GRE item-type scores. Figure 4-6 
illustrates the extent of unexplained variance (that is, the variance in GRE 
item-type scores attributable to sources other than the precollege abilities of 
the students, as measured by the SAT). Only the converted GRE sub-scores are 
graphed in Figure 4-6. The findings tend to agree with previous reports of this 
project and those by Wilson (1985) suggesting that GRE item-types may have 
greater correlation with variance in learning during the college years than do 
the GRE sub-score&. 

As has been previously discussed, critics of the GRE and SAT as measures of 
general learned abilities attack the validity of the measure themselves. These 
criticisms are based primarily on the use of the test sub-scores and the total 
test scores; the use of the item-type scores on either the GRE or SAT as 
multiple measurec of general learning have not been widely explored (Adelman, 
1988). The reliability of GRE item-types, their strong correlation with SAT 
sub-scores, and their apparent ability to measure discrete types of learning 
suggest that they may be of potential value as criteria in the assessment of 
general learning abilities of undergraduates. 
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For the purposes o£ this research^ precollege TOasures of student ability 
were defined as the SAT-V, SAT-M, and the total SAT score. Although there is 
research to suggest that SAT item- types may have greater predictive validity of 
college performance (Raniist, 1981a; 1981b; Schrader, 1984), SAT item-types were 
not available for use In the research model. Postcollege measvires of student 
learning were defined as the 9 item- type scores on the GRE. According to Astin's 
model (1970a, 1970b) ^ the effect of the learned abilities of students entering 
college on student outcome measures should first be determined* Once the 
variance in GRE it era- type scores attributable to SAT scores was determined, the 
une3Q>lain6d variance (score residuals) could be used as a proxy of change in 
general student learning along the 9 item- type measures . 

The SAT Verbal scores were used to predict each of the GRE Verbal 
item-types using the general lineal nK>del, The SAT Hath scores were used to 
predict each of the corresponding GRE Quantitative item-types. The cc^ined SAT 
scores (SAT Total) were used to predict the GRE Analytic Scores. The regression 
analysis summarized in Figure 4-^5 was performed by the PROC REG in the SAS 
statistical package • Individual student score residuals theoretically ranged 
from +1.00 to -1.00. Since individual scores were predicted relative to the 
combined sample group, some students will have higher actual scores than 
predicted scores, while others will have lower actual scores than predicted 
scores. A negative residual represents the unexplained variance of a student 
whose actual GRE item- type score was less than that predicted by the student's 
corresponding SAT subscore. Larger une3q>lained variance on a given item-type 
indicated larger ain>'>unts of change (either gain or loss) in general learned 
abilities associated with that measure. 

Variation in GRE item- type scores may be attributed to two sources: 
variation due to changes in the independent SAT variable and changes 
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attributable to other sources. In all cases in the Georgia State Native Grot^>, 
the prc^ility of obtaining the F value was .0001, suggesting that the general 
linear model is adequate in eaqjlaining the sources of variation within the C5RE 
item-type scores. R-squared represented the percent of total C5RE item-type 
variation eaQ>lained by the independent SAT variable. The residual variation in 
each GRE item-type— that not measured by the SAT score— was used as the proxy 
measure of general learned abilities during the undergraduate experience. These 
student residual scores, from the tijne they took the SAT prior to college to the 
time they took the GRE as graduating seniors on each of the 9 item-types, served 
as the measures of general learning among the Georgia State Native Group 
(Hanson, 1988; Pascarella, 1987). 

Fig 4-6. ChAxm ha €RE i ten-types 
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and GRE sub-scr^res 



V- Review of Literature on Catalocfues 
The purpose of the GSU Catalogue studies was to describe the intended 
curriculum in general education for undergraduates. By exonining the nature^ 
scope, and structure of the curriculum^ constraints to students' course- taking 
behavior may be identified. Furthermore^ the catalogue studies provided a 
ccffljparative basis for earamining what the enrol lji»nt patterns of students who 
showed gains in general learning were in relation to what the college intended 
the curricular patterns in general learning to be. Such an examination was 
admittedly a ccmparison of local requiren»nts (general education degree 
requirements) with national norms of general learning as measiired by the GRE 
examination . 

The GSU Catalogue studies began with a review of pertinent prior research. 
The issues and procedures identified in previous studies guided the inquiry. 
The review of selected research in which college catalogs served as sources of 
curricular data and analysis revealed three basic avenues of investigation. The 
first used catalogs to analyze the development of a specific academic area or 
topic. The second avenue of research employing catalogs studied change in the 
general education curriculum which occurred over a specific time period. The 
third research area involved the use of catalogs to develop course 
classification systems. 

Catalog studies of specific academic areas 

Becan-McBride (1980), Boy sen (1979), Fosdick (1984), LoGuidice (1980), 
and Tenopir (1985) used catalogs to study the status of particular academic 
areas. Two other studies (Gillespie & Cameron, 1986; Grave, 1985) used catalogs 
to analyze the development of a specific academic discipline over time. 
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Boysen (1979) analyzed the undergraduate technical cannnunications 
curriculum at selected colleges of engineering. Institutions were chosen for 
inclusion in the study based on the clarity of written course and program 
descriptions found in the college catalogs. Boysen' s study showed that many 
catalogs are replete with ambiguities and that generalizations of curricular 
cooiparability may be drawn only to those institutions with clearly written 
catalogs . 

Becan-McBride (1980) used the catalogs and brochures from medical 
technology programs to study characteristics such as course requirements, nature 
of *he program, degree earned, and accreditation. The study found tliat 
different types of medical technology programs mandated different prerequisites, 
thereby impacting on course- taking behavior evidenced on student transcripts. 

LoGuidice (1983) analyzed the catalogs of Lutheran-affiliated colleges and 
universities to determine the presence of a global/ international curricular 
perspective. Course titles and course descriptions were examined for certain 
key words and phrases selected a priori . LoGuidice found descriptive 
commonalities across like departinents or disciplines, with some phrases 
appearing more frequently in certain programmatic areas. 

The purpose of Fosdick's research (1984) was to determine the educational 
impact of information science on library science curricula. Catalog course 
titles and descriptions were used as data sources because Fosdick believed them 
to be more objective than verbal or written questxt^inaires. He found, however, 
several problems seemingly inherent in catalog resea'.xh: "... investigation of 
the catalogs does involve the subjective judgments of the surveyor, and a few 
schools do not provide adequate information in their catalogs on which to base 
an appraisal ..." (p. ?93). 
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Tenopir (1985) studied the kinds of college courseB used to determine a 
si>eciaIization in information science through an analysis of coiirse 
descriptions, course titles, and/or department titles. Based on this 
descriptive information, courses were assigned to one classification category. 
Originally selected to provide an undbtrusive means of data collection^ Tenopir 
found that the lack of ccsoion nomenclature in college catalogs irade the study 
difficult and time consuming. While many of the saro courses existed within all 
parent departments, most did not include the word "information" in their title 
making initial departrontal identification problematic. 

Grace's study (1985) examined the context of higher education 
administration curricula in order to determine the extent of profess ionalizat ion 
within the field. She used college catalogs from 1972-73 and 1982-83 to gather 
and COTipare information on the total nuiriber of program hours required, and the 
nuniber of reconanended courses within and outside higher education. Grace also 
included program handbooks and student transcripts in her data collection in 
order to be able to address the issue of intended curricula versus actual course 
taking behavior. 

Gillespie and Cameron (1986) used college catalogs, textbooks, and national 
convention programs to determine the developn^nt of the teaching of acting in 
colleges and universities over a forty-year time period. In order to delineate 
appropriate courses for inclusion in the study, Gillespie and Cameron chose only 
those courses in which the word "acting" or a variation of "acting" appeared in 
the course title or description. 

From these studies, it appears that college catalogs can be useful in 
examining the status of specific academic areas or their development over time- 
In each case, while other sources may have been included, the primary data were 
derived from an analysis of course titles and/or course descriptions found in 
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the college catalogs* Honever, the studies also revealcnl several weaknesses 
Inherent in the use of catalogs (particularly course titles) as priraary data 
sources. The airit)iguity of the catalogs thesffielves as irell as the varying 
vernacular intra- and inter institutionally make data gathering difficult and 
time consuming, and analysis more suJbjective than it at first appears « These 
issues are not mitigated by the use of a priori designations which, while common 
in the reviewed research, often necessitate judgmental categorizations. 
Multiple data sources including student transcripts may therefore be isportant 
to future curricular research, especially when attenuating to understand the 
intended versus actual curriculum. 

Studies of catalog change over time 

Four studies, l.-afferlin (1969), Dressel and DeLisle (1969), Blackburn, 
Armstrong, Conrad, Didham, and McKune (1976), and Toombs, Fairweether, Amey, and 
Chen (1989), used college catalogs to examine broader higher education 
curricular issues. Hefferlin studies curricular change based on e^ansion and 
reform over a five-year period. He used five areas of the catalog as data 
sources: program majors, areas of concentration, requirements within majors, 
degree requirements for graduation, and general curricular regulations, 
Hefferlin developed the Study of Institutional Vitality (SIV) model for 
analyzing curricular reform by calculating the proportion of courses that were 
dropped or noticeably changed in departjj^nts where the number of courses 
increased. Reform was also calculated by the proportion of courses added or 
changed in departments where the number of courses increased- Reform was also 
calculated by the proportion of courses added or changed in departments where 
the number of courses had decreased. Hefferlin found that expansion occurred 
more frequently in upper division courses than in lower division, and that 
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curricula at state colleges expanded more than at any other institutional type. 
The general education COTixjnent remained stable during the study period and the 
proportion of the curricula devoted to the academic major increased - 

The other three studies focused primarily on the status and changes in 
general education curricula over tin». Change was basically determined on the 
basis of breadth and depth, bj«9adth being defined as the percentage of coujrse 
requironents represented by general education, and depth as the percentage 
represented by the academic major. 

Dressel and DeLisle (1969) studied 322 college and university catalogs to 
determine change over the ten-year period from 1957-67. They found a decline in 
the specificity of general education requirenrents , a wide variation in the 
course and credit requirements for academic majors, and an increase in the 
proportion of elective courses permitted in fulfilling degree requirements. 
Part of this latter increase seemed tied to the simultaneous emergence of more 
individualized student learning experiences such as honor programs, advanced 
placement courses, and study abroad programs. 

The study by Blackburn et al. (1976) was, in part, designed to replicate 
the work of Dressel and DeLisle, examining the status of undergraduate education 
between 1967 and 1974. Using a slightly smaller sanple (271), this study found 
that the proportion of general education courses r€>quired for a degree decreased 
during this timeframe while the proportion of coursework devoted to the major 
changed very little. This resulted in a net gain in the elective course area. 
Using transcript analysis, Blackburn et al. went on to examine the course-taking 
behavior of students during this same tiiro period as cainpared to degree 
requirements evidenced in the catalogs. Again, though the sample included in 
the transcript analysis was very small, the study found a high level of 
congruence between stated degree requirements and student course-taking 
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behavior. The researchers concluded that there was a need to examine the 
relationships between different teaching techniques^ curricular programs, and 
different learning outcooms. 

Toombs et al. (1989) sought to show change in the general education 
curricula since the Blackburn study. Their research involved an analysis of 
data frost 700 1986-87 college and university catalogs including degiee, general 
education/ and major requirements as well as descriptive information found in 
curricular mission statwnents. The research showed an increase in the 
proportion of the curriculum devoted to general education since the 1976 study ^ 
while the total credits for a baccalaureate degree remained stable. The 
increase primarily occurred thrt>ugh an expansion of technical course 
requir«nents such as writing, speech, cOTputer and quantitative reasoning; 
little increase was found in humanities, social sciences, and nature sciences. 

Each of the preceeding studies acknowledged the difficulties in using 
catal^^gs as primary data sources described by Dressel and DeLisle in the 
1960's. Catalogs are often poorly organized, inaccurate, ambiguously written, 
and intra- and interinstitutionally inconsistent. Hefferlin's SIV nradel, 
Blackburn's transcript analysis, and Tooths' sairple size and descriptor analysis 
can all be seen as ways of methodologically inq^roving studies using catalog 
data. The use of transcript analysis, while making the roost viable connections 
between theory (catalog data) and practice (evidence of student course-taking 
behavior), also proved to be slow and expensive. 

Studies of course coding and classification 

Two fairly recent studies used college catalogs in the development of 
course classification systems at the state and national levels. Waggaman (1980) 
examined the effect of various forms of credit and non-'Credit designations on 



the Florida Statewide Course Nuaijer Systeai (SCMS). The significant variation 
that existed made it difficult to administer student credits when transferring 
from one Florida institution to another. Hagganon studied catalogs, various 
state and federal laws, and accreditation docua^nts from professional 
organizations. This analysis led to an examination of past and present credit 
practices in Florida as well as reconrondations for specific policies to reduce 
the existing problems in course classifications by level and department. 

On a national level, the Classification of Secondary School Courses project 
(NCES, 1982) used course descriptions collected from high school catalogs to 
develop a nationwide inventory of secondary school courses. A panel of 
reviewers examined the catalogs and established a course title index for each 
course, a unique 6-digit code number, keyword descriptors, and alternate course 
titles. The director was designed to be used by NOES for coding high schoi I 
transcripts. 

Discussion and results of the literature review 

Several themes emerge from this literature review relevant to the use of 
catalogs as primary data sources for studying the curriculum. The studies 
suggest that, when studying curricular change especially theory to practice, it 
is necessary to include multiple data sources. This n«y siJiqsly involve 
utilizing various sections of a college catalog, for exattfde mission statements 
and course descriptions in addition to course titles and degree requirements. 
Increasing the san?)!© size or using catalogs fran several academic years can be 
useful in studying breadth and depth of curricular issues. Student transcripts 
can add a comparative perspective between intended and actual curricula. A 
careful reading of the catalog must still be included, however, to identify 
prerequisites, number of courses required or reccaranended within and outside a 
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field of study—all of which effect course-taking behavior but which may not be 
easily differentiated when usin^ transcripts alone. The studies indicate that 
college catalogs, especially in conjunction with other dpta, can also be used to 
demonstrate change in academic departments^ general education^ and the 
proportional relationships of curricular caoponents at progranmiatic and 

institutional levels. 

From a methodological perspective, consDon research designs were used in the 
studies discussed above. Generally, these studies used predetermined categories 
of courses established de novo or derived from previous research. Sinple word 
counts or qualitative content analyses of course titles and/or descriptions were 
used to determine placeront within each category^ but there was no evidence that 
formal procedures were used to examine the content or concurrent validity of the 
categories themselves. Purthermore, ambiguities and inconsistencies in wording 
made it difficult to assign courses to predetermined categories or to id^mtify 
ccsnparable programs • In response, sate researchers erected more discrete and 
exacting operational definitions upon which to judge and classify courses. Even 
with these more stringent attempts, there remain questions as to the validity 
and reliability of classifications which depend on clearly vforded course titles 
and descriptions, as well as questions regarding inter-rater reliability in 
categorizing the data. Since a major attraction of catalogs as primary data 
sources has been their objectivity and accessibility, it is in?>ortant to 
recognize the researcher subjectivity inherent in collecting and analyzing 
catalog information. 
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structure and Content of General Education Requirc^ntB at GSU 

All the students in the Georgia State Native Saoples began and cCTfleted 
their educational program at Georgia State University. Undergraduate course%#ork 
at GSU is structured according to the institutional mission and by the 
baccalaureate degrees offered* The study of GSU catalogs trcm 1983-84 to 
1987*88 began with an examination of institutional mission and undergraduate 
degree requir^^nts. 

The Carnegie Classification schene pzrovides a means for placing all 
colleges and universities within a cca!i>rehensive categorization syst^ of six 
institutional types ("Carnegie Foundation^" 1987). According to this typology, 
Georgia State University is a public doctorate-granting institution. To fulfill 
its threefold mission^ it prooK^tes the advancement of knowledge through excel- 
lence in teaching^ research^ and public service. The institution's mission 
focuses on developing in students the requisites for con9>etence, personal 
fulfillment, and responsible leadership in business and the professions, in the 
sciences, in the creative and performing arts, in governn^nt and in public 
service. 

Georgia State University has five colleges that offer undergraduate 
degrees: Arts and Sciences, Business Administration, Education, Health Sciences, 
and Public Affairs. The University requires each student seeking the baccalau- 
reate degree to con^lete satisfactorily a general education cop^nent. Included 
in this section are the course numbers listed in the 1986-87 and 1987-88 GSU 
catalogs which constitute the requirements relative to the breadth of the 
curriculum (Blackburn et al. , 1976), The purpose of this part of the catalog 
study was to identify the minirauro general education requirements of each degree 
offered by the University* For each degree there are other requirements {listed 
as Areas IV, V, VI, and VII) which specify depth (major concentration) 
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requireanents which are not listed here. 
Distribution requirement 

The exact nature of the genercl education core requirement varies a«newhat 
by college and degree. This basic core of general education subjects includes 
60-80 quarter hours of coursework and is drawn from three customary groupings of 
disciplines: Area I: Humanities, Area II: Mathematics and Natural Sciences, 
and Area III: Social Sciences. All colleges conform to the distributional 
pattern of general education requirements, with minor variations, that follows: 
Core Area I~HD11MIITIES . Analysis of the humanities requireownts 
suggests a tendency to require courses or competencies in English 111, 
112, 113 to ccanplete the coraposition aspect of the Core Area I 
requirenwnt. Speech 150 and Philosophy 241 tend to be reccamnended 
although any 100 or 200 level English course can be chosen to cooplete 
the remainder of the requirment. The fine arts are also included to 
represent the practice and trends in the whole field of the 
humanities- Two other humanities options that appear are Art History 
170, 175, 180, and Music 161, 193/393. 

There is no requirement for demonstration of a proficiency level 
in a foreign language sequence, although the catalog offers classical 
and modern foreign language sequences in Arabic, Dutch, French, 
German, Greek, Hebrew, Italian, Japanese, Latin, Russian, 
Scandinavian, and Spanish. Reference to the Department of Foreign 
Languages accounts for additional courses that may be taken to fulfill 
the 20 quarter hour humanities requirement . 

Core Area II— MATHEMATICS AND NATORRL SCIEHCES. Only the Bachelor of 
Business Administration and the Bachelor of Social Work specify a 
requirement of mathematics and natural science to be met. A 
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prevailing pattern of a 10 -hour sequence in a laboratory science tends 
to accoT^ny a general natural science requir&nent. The widely 
varying practices exhibited in the choices of Mathematics and Natural 
Sciences courses are represented in selections from AstronCTiy 101, 
102, Biology 141, 142, Qiemietry 101, 102, Coonputer Science, Decision 
Sciences 104, 122, Geography 103, 104, Geology 101, 102, Mathematics 
107, 211, 212, and Physics 101, 102. There is some ambiguity as to 
what is included in the sciences. On this point, geography is 
included among the alternatives available in the social sciences 
distribution requirements. 

Core Are^: III — SOCIAL SdENCES. History 111 and 112 are prominent 
among the social sciences requirements, either an an alternative to or 
an equivalent of History 113. Political Science 101 is a requirement 
in the Bachelor of Social Work degree and Political Science 201 is 
required for the Bachelor of Business Administration. Occasionally 
specific reference is giv<an in the degree requirements to behavioral 
science electives. Among the available options in the social sciences 
distribution requiren^nts , two courses may be selected from the 
following: Anthropology 201, 202, 203, EconOTiics 210, Geography 101, 
205, Philosophy 101, 202, 203, 204, or Sociology 201, 202. 
The general education requirements attach some iJi?K)rtance to the level of 
the course. With the exception of the dual listed MUS 193/393 in the GSU 
catalog, no 300 or 400- level courses are listed- Figure 5-1 reveals either the 
increased availability of lOO-^level courses or a preference of students to 
complete their core requirements within the lOO-level. Requirements in Core 
Area II are primarily fulfilled through freshman level 100 courses, as the 
percentages reveal. In contrast, slightly less than two-thirds of the courses 
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identified for Core Area III, Social Sciences, requirCTients are sc^homore level 
in the GSU catalog, yet less than half of the courses taken are sophoaore 
level. There is an even distribution of Core Area I, Hunanities, requiresents 
between freshnan and sophooiore level courses in the GSU catalog, but again the 
preference is for the freshman level. Courses intended to develop students* 
quantitative abilities are clearly lodged in the freshinan year eaqperience, while 
courses intended to develop verbal abilities are distributed evenly across the 
lower division of the curriculum. No junior or senior level course is 
specifically identified as contributing to the general education of GSU 
students . 

Figure 5-1. Distribution of core areas by level of GSU transcript courses. 





Freshmen 
100 Level 


SophcaiKsre 
200 Level 


Total 
Courses 


Core Area I. 


532 


227 


759 


Humanities 


70.1% 


29.9% 


100.0% 


Core Axea II . 


591 


119 


710 


Mathematics & Natural Sciences 


83.2% 


16.8% 


100.0% 


Core Area III. 


642 


502 


1144 


Social Sciences 


56.1% 


43.9% 


100.0% 


Total 


1765 


848 


2613 




67.5% 


32.5% 


100,0% 



Variation and Change in the GSU Catalogs 
The final step in the GSU study involved looking for variance in the 
curriculum by examining catalogs from the 1983-84 academic year through the 
1987-88 academic year. The courses selected to represent the curriculum during 
this period were those unduplicated courses taken by the students in the Georgia 
State Natives Combined Sanq^le. The total number of courses examined was 1080. 
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For each course a determination was made as to whether the course app^axed in 
each catalog within the time frame indicated and a notation was made. Courses 
dropped from the curriculum and departmental code changes were also noted - 
Siit5>le frequencies of these changes were then tabulated. 

During the time indicated the Department code initials for four departments 
changed in the GSU catalog. In the 19E2-83 GSU catalog Drama (DRAM) courses 
became Theater (TH) or Film (FILM) courses. Information Systems (IS) courses 
became Craiputer Information Systems (CIS) courses in the 1984/85 GSU catalog. 
Also, it was noted that in the 1986-87 GSU catalog the Art courses were listed 
under sub- specialties of Art such as drawing, crafts, sculpture, etc. An 
example would be that a drawing and/or painting course now has a listing of DP 
rather than ART, even though the course is still offered through the Art 
Department- In the same catalog the dej>artmental codes for the college of 
Education also underwent a total revision. Vftien these changes occurred, it 
appeared that there was also an increase in the number of courses offered in the 
revised areas. 

Within the time frame under investigation, there were ninety-seven course 
title changes noted in the GSU catalogs. These were cases where the course 
listing and course number remained the same, yet the title was sin9>ly altered. 
It cannot be determined from a catalog study whether all title changes reflect 
true change in course content or whether the title was changed to more 
accurately reflect what was being taught in the courses. 

As Figure 5-2 illustrate, roost of the curricular change occurred during two 
years, 1984-85 and 1986-87. The total number of course additions and deletions 
represented no greater than 6.98 percent for any one year (1984-85) and no more 
than 20.63 percent cumulative change since the 1982-83 acadanic year. These 
findings suggest a small amount of variability in the curriculum over the time 
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period studied. Unlike the conclusions of earlier catalog studies (Blackburn et 
al. , 1976; Dressel and DeLisle, 1969; Hefferlin, 1969), the rate of curricular 
change at GSU during the period studied was not rapid. The assun?»tion that 
course content and curricular content remained basically stable over time 
appeared to be confirmed. 

Figure 5-2. Curricular change in GSU College catalogs, 1983-1988. 



GSU 


Course 


Course 


Total 


Net 


Total 


Percent 


Catalog 


Additions 


Deletions 


Change 


Change 


Courses 


Change 


1983-84 


30 


11 


41 


19 


1080 


3.80% 


1984-85 


65 


14 


79 


51 


1131 


6.98% 


1985-86 


20 


15 


35 


5 


1136 


3.08% 


1986-87 


29 


26 


55 


3 


1139 


4.83% 


1987-88 


8 


14 


22 


-6 


1133 


1.94% 


TOTALS 


152 


80 


232 


72 


1133 


20.63% 



Figure 5-3 reveals the changes in GSU departmental programs. Certain 
departments evidenced a high nximber of curricular additions. For example, 
CcMnputer Science (CSC), English (ENG), Journaliian (JOUR), and Music (MUS) 
departments had substantial additions as well as the most net curricular 
change. Given Hefferlin's hypothesis (1969) that high frequency of curricular 
change occurs in environments of institutional instability and that low 
frequency of curricular change occurs in environments of institutional 
stability, further research may be warranted to determine if the hypothesis is 
applicable to different levels of the organization such as departments. Did 
faculty and staff in the departments with a high degree of net change sense 
greater departmental instability and disarray in the years preceding the catalog 
changes? Did faculty and staff in ccai?>arably sized departments with low 
curricular chfinge experience greater levels of departmental stability? Such 
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questions are beyond the scope of the current research project, but svay warrant 
investigation by other researchers. 
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Figxxre 5-3. Curricular change by GSO departiaent 

Dept Additions Deletions Total Changes Wet Change 

Code Hum % Num % Num % Num % 
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Findings and conclusions trcm the GSU catalog study 

Like previous studies (Boysen, 1979; Dressel & DeLisle, 1969; Fosdick, 
1984; Tenopir, 1985), the GSU catalog study encountered difficulty in placing 
courses into the predetermined categories and identifying like programs. Often 
judgments needed to be made on the basis of catalog descriptions as to whether a 
course had undergone substemtial revision^ was equivalent to a course appearing 
in a previous catalog under a different number and/or deparfcownt/ or was 
cross-listed with a course in another departn^nt. It was also ambiguous to 
whether certain courses were reccamended or required to meet the general educa- 
tion core requirements of the various degrees at GSU College- In shorty 
ambiguities in wording and phraseology made the catalog analyses difficult • 
Research on inter-rater reliability in such catalog analyses is needed. 
However, the catalog study was necessary and essential in order to ccmqpare and 
contrast the similarities and differences between the intended general education 
(as stated in the catalog) with the actual general education of students (as 
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evidenced on their transcript. 

The review of the prior catalog studies identified several B»tho^Iogical 
and substantive issues. First, prerequisites ^>peared as variables posing 
potentially significant constraints on course-taking behavior (Becan-McBride, 
1980). The GSI3 Catalog Study indicated a low percentage of course prerequisites 
had been ignored. This suggested that students were constrained in their 
course-taking behavior by cotirae prerequisites. 

A second issue discussed in the prior catalog studies was that course 
content nay exhibit carnK'talities across institutions by academic department or 
program (Dressel and DeLisle, 1969) and among institutions by type 
(Becan-McBride, 1980; Boysen, 1979; LoGuidice, 1983; Tenopir, 1985). However, 
general education requirements may be subject to broader social and intellectual 
trends and may vary regardless of institutional type < Dressel and DeLisle, 
1969). These observations required interinstitutional con?>arisons of general 
education requirements and of the content and distribution of courses within 
departments. Such analyses require can5>arable cataloq studies of the other 
institutions participating in this project, since there is but one institution 
per Carnegie classification ("Carnegie classification", 1987; Ratcliff, 1986), 
con^arisons of catalog content within a given type of institution were not 
possible within the scope of the current research project. 

Prior literature had suggested that the number of hours required of a 
specific program, the number of recosmiMnded and the nuinber of required courses 
within a field of study and outside a field of study were also found to be 
potential constraints on study course selection and enrollment (Dressel and 
DeLisle, 1969; Grace, 1985). The general education requirements for the 
baccalaureate degrees offered by the colleges of GSU varied considerably in 
their prescript ivity, suggesting that course enrollment patterns may vary by 
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degree and college. Such variation and its iiapect on GRE it«n-type residual 
scores will be explored in subsequent reports. 

Specific subjects or departments wera identxfied in the GSU general 
education requircaronts for one or ii»re of the bachelors degrees. The courses 
within the Departioent of Math«natics were generally prescribed or recamaiended to 
i^et Core Area II of the GSU general education requiredsient . The inclusion of 
niatheanatics courses in coursework patterns foiuid by cluster analysis of student 
transcripts not only tended to confirm the prescription of mathsanatics courses , 
but also suggested which of those courses students who perforxmd well on the GRE 
quantitative item- types enrolled. 

There appeared to be soa^ contradictions within the GSU general education 
requirements. Specifically, PHIL 201 and AH 170 and AH 175 were applicable to 
Humanities (Core Area I) in one instance and to Social Science (Core Area II) in 
another. Appearance of these courses within coursework clusters associated with 
high gains in verbal abilities would affirm the notion that students choosing 
these courses often also perform well on those sections of the GRE traditionally 
associated with the Humanities, Similarly, ap^arance of these courses in clus- 
ters related to analytic reasoning may suggest relationships with either the 
Humanities or Social Sciences. 

Particular value was placed on laboratory science courses taken in a 
sequence in meeting the GSU general education requirements for Natural Science 
(Core Area II). The appearance of these specific laboratory course sequences in 
the coursework patterns will be examined. 

The GSU Catalog Study revealed that the intended undergraduate general 
education was primarily (but not exclusively) confined to freshman and sophooore 
(100-200 level) courses- Analysis of the distribution of courses by level 
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appeared warranted for the couraeirork patterns analysis. 

The above findings and c^servations frcan the GSU Catalog provide a basis 
for making coB9>arison8 between the intended curriculum and the enrollment 
patterns of the GSU Native Coirbined Saa^ples, In the following Chapter, the 
results of the quantitative cluster analysis of GSU coursework is presented. 
The results are then contrasted with the findings of the GSU Catalog 
Study . 
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VI. 9"^"^^^°^^^^ Cluster Analysis of 

Georgia State Native Ccaiobinea Colleg e Saatple 



Overview 

This section reports the use of the quantitative cluster analytic procedure 
to analyze the Georgia State Native Group. The findings from the analysis of 
the combinea Sample fll and Sample «2 Native students is presented. The objects 
of these analyses are the courses which constitute the enrolljnent patterns of 
students in the Georgia State Native Group. The demographic profile of these 
Georgia State Native students was presented in Section 3. In Section 4, the 
distribution of GRE and SAT scores was presented. Also in that Section, some 
basic information on the distribution of courses on the students' transcripts 
was also presented. The subject of the overall research is the coursework in 
which students enrolled, not the students themselves. The criterion variables 
in the research are the GRE item-type residuals. The distribution of those 
residuals among the coursework is described below. 

Georgia State Native Student Group 

There were 7,850 courses listed on the 168 transcripts of the students in 
the Georgia State Native Group, indicating that, on average, each of these 
students had enrolled in 46.7 courses as part of the baccalaureate degree 
program. There were 1,244 unduplicated courses on the Native transcripts, 300 
in which 5 or more students had enrolled. These 300 courses were the objects of 
further analysis. Certain departments predominated in those enrollment 
patterns. Looking at the total (duplicated) course count of 7,850, 25 
departments had their courses appear 100 times or more on the student 
transcripts. The 100 or more duplicated courses were taken from each of the 
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following departments: 



Departront 


Course Count 




Code 


( LAipilCEue ; 




AC 


i41 


Accounting r 


ART 


153 


AT L / 


BIO 


241 


Biology ^ 


CHOI 


loo 




CIS 


112 


uoQ$>Uwer iniomiatjLoii byswenis^ 


CJ 


133 


Criminal Justice^ 


DM 


120 


Decision MatnenaticSr 


DS 


178 


Develc^m^ntal Studies^ 


DSC 


144 


Decision Sciences, 


EC 


263 


Econc»idc6 ^ 


ENG 


765 


English, 


FR 


119 


French , 


HIST 


375 


History, 


IS 


100 


Information Systems^ 


JOuR 


Ibb 


Journalism^ 


WATH 




nat.neznat.ics , 




TIC 


nanagesitent , 


NX 


167 


Marketing^ 


KUS 


352 


Music, 


PHIL 


176 


Philosophy, 


POLS 


277 


Politics, 


PSY 


301 


Psychology , 


soc 


260 


Sociology, 


SPAN 


129 


Spanish, 


SPCH 


120 


Speech . 



English was clearly the departinent of most frequent enrollment. 

Figure 6--1 shcMi^s that the lower division (61.20%) leads the upper division 
(38-80%) in enrollment, as denK^nstrated on the transcripts. 

Figure 6-2 shows that the courses on the transcripts were taken by some 
students over a period of years. Twenty^-two percent of the students enrolled in 
coursework from 1970 to 1981. These students were non- traditional and likely 
proceeding on a part-time basis. 
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Figure b-l. Distribution of courses by course numbering 

Cumulative 

Course level Frequency Percent Percert 



100-199 3,141 40.01% 40.01% 

200-299 1,661 21.18% 01.20% 

300-399 1,684 21.45% 82,65% 

400-499 l,34i 17.08% .^9.73% 

500-599 21 .?7% 100.00% 

TOTALS 7,850 100 00% 100 00% 

Figure 6*2. Year and Semester of Enrollment: Georgia State Native Group 
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Figure 6-3. The distribution of GRE iteoi-type residuals for 1^244 unduplicated 
courses in Georgia State Naxiive College Group 



GRE Item-types Number Max Min Score Residual Std Error Std 

of Itans Value Value Range Means of Mean Deviation 
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Figure 6-4, The distribution of GRE item-type residuals for 300 Georgia State Native 
courses used in the Qualitative Cluster Analytic Procedure (Procedure 1). 

GRE Item-types Number Max Min Score Residual Std Error Std 

of Items Value Value Range Means of Mean Deviation 
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Fig 6-5. Conparison of siandard deviatyin 
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GRE i ten-type residuals 



For each course taken by the Georgia State Native Group, the mean of GRE 
item- type residuals for students enrolled in the course was calculated- As 
means for GRE residuals for all courses and for courses enrolling 5 or more 
students were calculated^ certain trends eMrged. Examination of Figures 6-3 to 
6-5 show that as student data were aggregated according to courses enrolling 5 
or more students^ the standard deviation of these means was considerably smaller 
than those for all courses. This trend suggested that there are relationships 
between the coursework taken and the student score gains on the tests. 
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Cluster Analysis of Georgia State Native Grcyup 
Creating the raw data matrix and the re8eid>lance TOtrix 

Using the n^an residuals of the Georgia State Native Group and the 300 
courses found on 5 or xnore of the Georgia State Native student transcripts^ a 
raw data matrix was created. The data matrix consisted of 300 columns and 9 
rows (300 X 9). The rows represented the criterion variables: the 9 GRE 
item-type residual scores. The colunms represented those courses enrolling 5 or 
more students. Thus, each cell value of the matrix was a mean GRE item- type 
residual score for those san?>le group students enrolling in a specific course. 

A reseanblance matrix was created next to describe how closely each course 
resembles the other 299 courses according to the criterion va^-iables: the 
student residuals. To calculate the resenblance matrix, the correlation 
coefficient was selected as a similarity measure. The correlation coefficient 
was the Pearson product-moment correlation coefficient. Thus, this coefficient 
assesses a pattern similarity of any two courses explained in terras of the 9 GRE 
item' type residuals 

The resemblance matrix produced in this step consisted of 300 rows and 300 
columns (300 x 300), in which each cell value theoretically ranged from -1.00 to 
1,00. The calculation of the resemblance matrix was done using the SPSSx PROX- 
IMITY program. The program provides 37 different proxijDri.tY measures. Ten of 
them are for quantitative data and the remainder are for binary or qualitative 
data. This program can directly produce distance, dissimilarity, or similarity 
matrices as text files for a small to moderate number of cases and variables, 
which can be directly used for other SPSSx procediires or for other statistical 
programs. BMDP PIM can also be used to calculate the correlation resemblance 
matrix and save it as a text file. 
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Selection of the luster inq method 

The method selected for this quantitative analysis was the average lin)cage 
method (OTOIA). The appropriateness of the Average Linkage Method was discussed 
in the Task #4 Report of this project (Ratcliff, 1989). Contrary to reports by 
Romesburg (1984), UPGMA is available in SPSSx, SAS, and BMDP. All the three 
statistical packages provide UPCaiA as an option of the cluster program and use 
an identical method to coni>ute distances between clusters. In this analysis, 
the UPGMA method in the cluster program of SPSSx was eniployed for the sake of 
convenience. The original dendrogram of the Georgia State Native Group courses 
produced by SPSS-X is presented in Figure 6-6. 

Courses were classified into 13 coursework patterns according to a 
hierarchical cluster structure. In fact, the choice to present the data in 13 
clusters is arbitrary. Any nun4>er of clusters can be identified depending on 
the hierarchical cluster structure produced; this structure remains constant 
regardless of the number of clusters used to form coursework patterns. A 
procedure for selecting the optimum number of clusters and for validating the 
resulting patterns will be described in greater detail in a subsequent section 
on the discriminant analysis of the coursework patterns in the Georgia State 
Native Group. 

using a 13 cluster solution to the quantitative cluster analysis, the 
largest number of courses are found in Coursework Cluster #1 with 53 courses and 
Cluster #6 with 50 courses. The smallest clusters are the 13th cluster with 3 
courses and the 8th cluster with 4 courses. Overall, the differentiation 
between clusters is attributable to the number of criterion variables used in 
the analysis and also to the choice of those variables. The cluster analysis 
and subsequent discriminant analyses suggested that student residuals on GRE 
item-types are strong, reliable ".'■.^i robust measures in differentiating student 
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general learned abilities. 

The hierarchical cluster structure is presented in the dendrogram suomary 
of Figure 6-7 • For concise visual presentation, the caa;>lex sub-structures of 
each of the clusters were omitted fraoi the dendrogram. The dendrogram displays 
the clusters being coiiA)ined and the distances between the clusters at each 
successive step, suggesting that the IB-^cluster solution examined is appropriate 
and interpretable . Cluster analyses using smaller and larger numbers of cluster 
groupings provided comparably high levels of correct classification, as deter- 
mined by subsequent discriminant analyses. However, as the res^nblance index 
increases (Euclidean distance between courses), more distant courses are joined 
into larger and larger clusters. A 5-cluster solution, for exasple, provides a 
high degree of aggregation which may prove to have a high degree of predictive 
validity but a low level of utility in differentiating coursework by item^type. 
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Figure 6-6. SPSS-X Dendrogram. 
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Figure 6--6. SPSS-X Dendrogram. 
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Figure 6-6. SPSS-X Itendrogram* 
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Figure 6-6. SPSS-X Dendrogram. 
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Figure 6-6. SPSS-X Dendrogram. 
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Figure 6-6. SPSS-X Dendrc^am* 
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Figure 6-7. Dendrogran of hierarchical cluster structure— Georgia State Natives 
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Figure 6-8a. Courses within coursework clusters (13-cluster solution) : Natives 
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Figure 6'-8b 
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following a course indicates a course misclassif ied according to the dis- 
criminant analysis of course clusters. 
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Figure 6-8c. Couxses within coursework clusters (13-cluster solution): Natives 
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following a course indicates a course misclassif ied according to the dis- 
criminant analysis of course clusters. 



Observations about the Clusters 

A careful examination of courses within each cluster seems to indicate that 
some courses coming from the same department appear in the same cluster, such as 
the English courses (ENG) in Cluster #1, the Coogsuter Information System (CIS) 
courses in Cluster #2, and the Journalism (JOURN) courses in Cl. aster #7. 
Similarly, there are apparent sequences of courses, such as Anthropology 201, 
202, and 203 sequence in Cluster SI. Also, a set of courses coming from various 
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related disciplines may form a hanogeneous cluster on the basis of a set of 
given attrilnites or criteria The honwgonity of disciplines is particularly 
^parent in Cluster wl. Nevertheless, the huisanities and social sciences seem 
to predestinate this group. 

At this point in the analysis, it is difficult to describe which diaiensions 
of student general learned ability each cluster represents. However, it seeons 
to be clear that one pattern of course enrollment may contribute to student 
general learned ability in a way significantly different from the other course- 
work patterns. Supporting this is a more detailed examination of subset courses 
of each clusters. In many cases, those courses offered at the same level tend 
to be combined into pairs together. But, those pairs are agglomerated with 
other courses offered at the higher level again according to the hierarchical 
structure of clusters. This may suggest that student gains in general learned 
abilities may be obtained through a sequential enrollment pattern during the 
college years, not at a single stage of the se<iuence (such as the freshman year 
experience) . 
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Discrljninant analysis of courae work patterns 

In examining the dendrogram of the Georgia State Native Group, a logical 
question arises as to which number of clusters or pattern groupings provides the 
best explanation of the relationship between student item-type score gains and 
coursework patterns. Separate discriminant analyses of different nuinbers of 
cluster groupings were be performed in order to determine the number of 
groupings that optimizes the proportion of courses correctly classified. Four 
different cluster solutions provided coiJ5>arably high levels of correct 
classification: 

8 cluster solution : 81.67% of courses correctly classified 
11 cluster solution : 83.00% of courses correctly classified 
13 cluster solution : 81.33% of courses correctly classified 
15 cluster solution : 80.33% of courses correctly classified 

While these cluster solutions produced crarparable classification results, 
the different grouping evidenced differing effectiveness in identifying 
relationships between mean item- type residual scores and coursework patterns. 
The 13 -cluster solution was ustd in this report. 

The discrijninant analysis was conducted using tha DISCRIMINANT program in 
SPSSx in the following manner. Using the course item-type attributes as inde- 
pendent variables and the cluster group membership as the dependent variables, 
discriminant functions were applied to the data. The discriminant functions and 
the courses it em- type mean residual scores were used to see how correctly the 
discriminant function identifies each cluster group. The resulting percentage of 
correct predictions serves as a secondary validation of the cluster solution 
(Bradfield and Orloci, 1975; Green and Vascotto, 1978; Romesburg, 1984). 
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Figure 6-^. DiscrislaaDt aoalysis of the 13*clB8t«r solatlas for Geoxgla State Hatim Sample 



Acttal Mo. of Prodlcted firoi? Tlrtr iiililii 

QBrt^ Ca^ 9rl Qir2 _ Gr 3 6r 4 6r 5 6t 6 9r 7 te8 Sr 9 6r 10 11 6r 12 Sr 13 

Gns9> 1 53 48 3 0 0 0 0 0 0 0 01 10 
90.6% 5.7% .0% .0% .0% .0% .0% .0% .0% .0% 1.9% 1.9% .0% 

Qnap 2 49 0 45 1 0 0 0 0 0 3 0 0 0 0 
.0% 91.B% 2.0% .0% .0% .0% .0% .0% 6.1% .0% .0% .0% .0% 

Ota^ 3 15 00120100000200 
.0% .0% 80.0% .0% 6.7% .0% .0% .0% .0% .0% 13.3% .0% .0% 

Qtxap 491206000000000 
11.1% 22.2% .0% 66.7% .0% .0% .0% .0% .0% .0% .0% .0% .0% 

Gro«5) 5 40 2 2 0 1 31 10 1 0 1 1 00 
5.0% 5.0% .0% 2.5% 77.5% 2.5% .0% 2.5% .0% 2.5% 2.5% .0% .0% 

Groq> 6 50272003800001 00 
4.0% 14.0% 4.0% .0% .0% 76.0% .0% .0% .0% .0% 2.0% .0% .0% 

Group 7 14 01000013000000 
.0% 7.1% .0% .0% .0% .0% 92.9% .0% .0% .0% .0% .0% .0% 

Gmap 84200 0000200000 
50.0% .0% .0% .0% .0% .0% .0% 50.0% .0% .0% .0% ,0% .0% 

ijroup 9 19 03001000 15 0000 
.0% 15.8% .0% .0% 5.3% .0% .0% .0% 78.9% .0% .0% .0% .0% 

Group 10 12 030 0000009000 
.0% ».0% .0% .0% .0% .0% .0% .0% .0% 75.0% .0% .0% .0% 

Grotqj 11 130200000000 11 00 
.0% 15.4% .0% .0% .0% .0% .0% .0% .0% .0% 84.6% .0% .0% 

Group 12 19 1 20 020000 1 0 13 0 
5.3% 10.5% .0% .0% 10.5% .0% .0% .0% .0% 5.3% .0% 68.4% .0% 

Graq) D30200000000001 

.0% 66.7% .0% .0% .0% .0% .0% .0% .0% .0% .0% .0% 33.3% 



PtttxwDt of "Groiqjed" Clusters correctly classified: 81.33% 
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Correlations of itan-tvpes and discrimina nt functions 

The discriminant analysis of Georgia State Native Group provided secondary 
validation that 81.33% of the classification of courses was correctly predicted 
by the cluster analysis (See Figure 6-9). The discriminant analysis is a 
secondary validation, since it is based on the sane saicple of transcripts and 
test scores. 

Stated siit?)ly, approximately 8 of 10 courses roost frequently taken by 
Georgia State Native students were correctly classified according to their mean 
residual GRE scores. While the cluster analysis produces coursework patterns 
according to criteria of general student learning, additional steps are needed 
(1) to determine which courses were correctly classified and (2) to ascertain 
which item- type residual scores contributed to any given coursework pattern. 

Using the BREAKDOWN procedure in the DISCRIMINANT program of SPSS-X 
(Norusis, 1985), courses which were incorrectly classified or which may be 
classified within another coursework pattern are identified. These courses are 
marked with an in Figure 6-8. When a set of data is taxonomized by multiple 
and independent criteria, it is reasonable to expect that the first groupings 
are generally the most homogeneous and the later groupings are the most 
heterogeneous. Consequently, the highest proportions of misclassif ications 
occurred in Clusters #4, «5, #6, #8, #12 and #13, while Clusters #1 and #2 had 
relatively few misclassif ications. 

To compute the contribution of each mean item- type residual score to the 
discriminant functions, the correlation coefficients between mean residual 
scores and discriminant functions were examined. Figure 6-10 shows the pooled 
within-group correlations for the 13-cluster solution of Georgia State Native 
Group coursework. 
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Figure Fooled vithlfi^gcmp comOsticM) betvom aan ita»<ype c«d&al ^oras esoA aiscrifldnaiit 
ftmcXiaas for 08csgia State latlve Snoop. 





FCBIC 1 


FiBc 2 


Fnoc 3 


fXSDC 4 


fmc 5 


fnnc 6 


fmc 1 


FtBK 8 


Fwc 9 


Inalytic Baasonliig 


.6134 * 


.3644 


.0083 


.5712 


-.1139 


.1953 


.2710 


-.1191 


-.1609 


Quantltotive CoefgriBOos 


.0394 


-.1567 


.4340 




.66S1 • 


-.0108 


.3045 


-.0037 


-.0609 


Begular XatbeBatice 


.3521 


-.1719 


.4048 


.2269 


-.1873 


.6S39 • 


.2927 


-.1688 


.2331 


Reading CaBfaeSnslffl 


-.1966 


.5880 


.6441 « 


.3851 


.0162 


.1140 


.1968 


.0225 


-.0012 




_ cam * 




"•/to!) 


.32a6 




.1461 


.2833 


-.lUZZ 


.4205 


Sent^HSB CGopletiOQ 


-.U05 


.0860 




.1562 


-.3054 


.1186 


.681fi • 


.0213 




Analogies 


.0781 


-.^10 




.3301 




-.4333 


.6081 * 


-.1268 


-.1293 


Dkata IfsterpzetatiGD 


.1369 


.0964 


.0134 


-.0494 


.1303 


.0587 


.4153 


.7576 • 


.4500 


Logical Sessoning 


.0316 


.1246 


.0272 


-.3969 


.2784 


-.1853 


.7409 • 


-.4018 


.0545 



IfcJte. Nbbd reslilnl Iteertjpe aonrelatiaffi cf^red by slxe vlthlii fioctloa. Heon IteB-typs residiiaLs with 
large coef fidoits are pareseoxted is bold, graoped togetbei: aid are natkad with <»«n^»r1f4ny (*) . 



Correlations of coursework clusterB and discriminant functions 

Figure 6-10 svanmarizes relationships between GRE itan-type residuals and 

discriminant functions: 

Function 1 was negatively correlated with Antonyms (r=-.62), and 

was positively correlated with Analytic Reasoning (r=.61); 

Function 2 was positively correlated with Reading Ccacprehension (r=.59); 

Function 3 was positively correlated with Reading Conqprehension (r=.64); 

Function 4 was positively correlated with Analytic Reasoning {r=.57), and 
was positively correlated with Quantitative Ccanparisons (r=.50); 

Function 5 was positively correlated with Quantitative Comparisons (r=.67); 

Function 6 was positively correlated with Regular Mathanatics (r=.65); 

Function 7 was positively correlated with Analogies (r=.61), 

was positively correlated with Sentence Conqsletion (r=.68), and 
was positively correlated with Logical Reasoning (r=T74); 
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Function 8 was positively correlated with Data Interpretation (jr=.76); 

Function 9 was negatively correlated with Sentence Cc»rpletion (r=-.57)- 

The pooled within-group correlations established relationships between the 
discrifliinant functions and the GRE itera-type residuals. Each discriminant 
function explains a certain proportion of the variation in residual scores. 

Discriminant functions with strong explanatory power ^ "good discriminant 
functions/' have large between-cluster variability and low within-cluster 
variability (Haggerty^ 1975; Norusis^ 1985). The eigenvalues of Figure 6-23 
present the ratio of between^-group to wi thin-group sums of squares of the 
residuals. Large eigenvalues are associated with the discriminant functions that 
most contribute to explaining variability in GRE item-type scores. 

Wilk's Lambda is the ratio of the within-group sum of squares to the total 
sura of the squares. It represents the proportion of the total variance in the 
discriminant function values not esq^lained by differences among cluster groups* 
Wilk's Lambda serves as a test ox the null hypothesis that there is no 
difference in the mean residuals of a coursework cluster means and the mean 
residuad scores of the coursework in the tot€Ll san^le. 

Thus^ the eigenvalues and canonical correlations indicate the extent to 
which each discriminant function contributes to our understanding of the 
variability in coursework mean residuals. Lambda test the null of the 
differential coursework hypothesis for each discriminant function. 
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Figure 6-11, Canonical discrijaflinant functions: Georgia State Native Group 
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Flgnre 6-12. Cffixmical diSGrloinaDt fmKrtioBS evaloBtsd at groop 



Uuster Fmc 1 fmc 2 Ftac 3 Txax 4 Pane 5 Ftioc 6 fme 7 Fmic 8 Fn»; 9 HialiiiB BaxiflBn Average 



austeril -1.3ST9 .9989 ,1560 -.8726 . 3073 -.3984 -.0452 .4111 -.1456 -1.3579 . 9989 -.1052 

CloBtar 12 1.1563 .8198 .1260 -.0312 -.2500 -.3509 -.31K -.3162 -.1454 -.3509 1.1563 .0766 

Cluster 13 -.1258 -.6376 -1.0305 1.4587 -•5443 ,1301 .8223 -1-0257 .1379 -1.0305 1.4587 -.0905 

Cluster #4 -.um 1.0111 -1,3948 , 3299 1.1899 -.9433 -,0267 -.6102 4132 -1.3948 1.1899 -.0123 

Cluster #5 -1.6043 -1-4015 .3904 .6182 -.3567 .4197 -.0095 .0475 -.3693 -1.6043 .6182 -.2517 

Cluster »6 1.1021 -.9841 -1.0649 -.5749 .0788 .6357 .5798 -.1121 .4763 -.9841 1,1021 ,1263 

Cluster «7 1.2354 -.4306 -,1244 -.7350 1.6325 -.6735 -1.9183 -.2743 ,5885 -1.9183 1.6325 -.0777 

Cluster #8 -.5116 -.5367 .6794 -.9594 -.8844 .1355 1.2345 .5577 -.5681 -.9594 1,2345 

auster 19 -.2378 1.4366 -.2678 1.5730 -,S743 1.3124 .7080 -.2496 -.3560 -.3743 1.5730 ,3927 

Cluster «10 . 5220 -1.0595 1.3137 -.0040 -1.1973 .0662 -.2229 1.0360 -,4350 -1.1973 1.3137 ,0010 

Cluster #11 .8406 -.8109 -.2390 1.0540 1.48M -1.6430 .6189 -.1521 .5261 

Cluster m -.06» ,8662 2.1906 .0263 -.M40 -.0995 -1.06B1 .8303 .0374 

Cluster #13 .0818 .1609 .:429 -.9916 -,4671 .8943 .2294 -.4885 -,5079 



-1.6430 1,4889 .1871 
-1,0681 2.1906 .2319 
-.9916 .8943 -,0829 



HiniBUB -1.6043 -1.4015 -1.3948 -.9916 -1.1973 -1.6430 -1.9183 -1,0257 -,5681 
naxiauB 1,2354 1,4:^ 2A90b 1.5730 1.6325 1,31:M 1.2345 1.0^ .5885 

Average .0731 -,0444 .1596 .0686 -.OOOi -.0396 .0449 -.0274 -.0268 
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Interpreting the coursework clusters for the 13>-clus ter solution 

Figure 6-12 shows the coursework cluster means (groiq) centroids) for each 
discriminant function, clusters with positive or negative means greater than 
1.0 were selected for further analysis. 

Coursework clusters with positive or negative means greater than 1.0 were 
selected for further analysis. Coursework Cluster #1 had a high negative group 
mean on Function 1 and a high positive group mean on Function 2. Function 1 was 
positively correlated to J^alytic Reasoning (r=.61) and was negatively 
correlated to Antonyms (r=-.62). Function 2 was positively correlated to 
Reading Con?)rehension (r=.59). Students enrolling in this coursework iinproved 
in Antonyms and Reading Comprehension but declined in their Analytic Reasoning 
abilities. 

Cluster #2 had a high positive group mean on Function 1. Function 1 was 
positively correlated to Analytic Reasoning (r=.61) and was negatively 
correlated to Antonyms (r=-.62). Students enrolling in this cluster gained in 
Analytic Reasoning but declined in Antonyms. 

Cluster #3 evidenced a high positive group mean on Function 4 and high 
negative group mean on Function 3. Function 4 was positively correlated to 
Analytic Reasoning (r=.57) and Quantitative Ccsnparisons (r=.50). Function 3 was 
positively correlated to Reading Coii^jrehension (r=.64). Students taking Cluster 
#3 coursework improved in Analytical Reasoning and Quantitative Comparisons but 
declined in Reading Conqsrehension. 

Cluster #4 had high positive group means on Functions 2 and b, and a high 
negative group mean on Function 3. Function 5 was positively correlated to 
Quantitative Conqparisons (r=.67). Students enrolling in this cluster showed 
gains in Quantitative COT5>arisons. The results for Reading Ccstiprehension were 
inconclusive . 



ERJ.C 



- 130 - 



Cluster iS had high negative group means on Functions 1 and 2. Students 
enrolled in this coursowrk gained in Antonyius but declined in Analytic 
Reasoning and Reading Ccoprehension. 

Cluster §6 encooimssed high negative grcnip n»ans on Functions 2 and 3, and 
a high positive group mean on Function 1. Students signed up for this 
coursework pattern declined in Antonyms and Reading Catq)rehension but gained in 
Analytic Reasoning. 

Cluster #7 had high positive group means on Functions 1 and 5. Students 
taking this coursework pattern gained in Analytic Reasoning and Quantitative 
Co8i^>arisons and declined in Antonynss. 

Cluster #8 consisted of three courses. Two courses were misclassif led. 
Therefore, no further analysis was conducted with this cluster. 

Cluster #9 had high positive group means on Functions 2 and 4. Students 
enrolled in these courses in^^roved in Reading Cc^rehension^ Analytic Reasoning, 
and Quantitative Coanparisons . 

Cluster #10 had high negative group means on Functions 2 and 5, and a high 
positive group mean on Function 3. Students enrolling in these clusters showed 
declines in Quantitative Comparisons. The results for Reading CaDqp>rehension 
were inconclusive. 

Cluster #11 enccHtQ>assed high positive group means on Functions 4 and 5. 
Students registering in this coursework gained in Quantitative Comparisons and 
Analytic Reasoning* 

Cluster #12 had a high positive group mean on Function 3. Students taking 
courses in this cluster improved in Reading Con^jrehension. 

Cluster #13 consisted of three courses* Two courses were misclassified. 
Therefore, no further analysis was conducted with this cluster. 




Figures 6-13a — 6-13b> portray the coursework clusters and the nean residual 
it«a-type with which thoy were found to be associated. It should be cautioned 
that the association was established at the cluster level. No direct causal 
link is intimated between student enrollaaent in any one given course and scores 
on the GRE. Furthermore, at this point, one cannot say why students who 
enrolled in these courses had higher score gains. The cluster serves to 
hypothesize relationships between coursework patterns and the general learned 
abilities meastires by the item-types of the GRE- One can oay that stotents who 
enrolled in specific patterns of cxfuraework t^Kled to evidem» strw^or gains on 
specific itesD- types within the GRE, while others irtio enrolled in different 
coursework patterns did not tei^ to show such gains, nils evideiure affiins the 
hypothesis that stuctent gains in general learned abilities are associated, 
positively and negatively, with the co urs ework in which they enrolled. Further 
analysis is required to determine the nature of these associations. 
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Figure 6- 13a. 



Cluster 1: High positive mean residualfi on Antonyns (ANT) and 
Reading OaprehenBion (BD); high negative mean residuals c> 
Analytic Reasoning (ARE) 
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Principles of Ac«>unting I 
Interrodiate Acccnintii^ I 
Introduction to Physical Anthropology 
Introduction to Cultural Anthropology 
Introduction to Archaeology 
Business Law (first course) 
Criminology 

Structure of Music Industry 
Decision Mathematics XII 
History of the Motion Picture 
Develoi^ntal Mathematics II 
Economics of Urban Problons 
Dynamics of Labor Prc^lems 
Eurc^an Literature II 
The Short Story 

Major American Writers 19th/20th Century 

Narrative Technique/ Creative Writing II 

Poetry Techniques/Creative Writing III 

Mc^ern British Literature 

Exceptional Children/Youth 

Prtd>l«ns of Urban Education 

Intermediate French 

Intermediate French 

Physical Geology 

World Civilization Before 1500 

World Civilization Since 1500 

Principles of C«ii>uter Programming 

Introduction to Data Processing 

Problem Analysis: FORTRAN Programming Language 

Principles of Coirputer Programming 

Elementary Italian 

News Writing and Reporting 

Advanced News Writing and Reporting 

Elementary Latin 
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Introduction to Music 

Introduction to Philosophy 

History of Western Philosophy I/400BC-1300AD 

Physics of Music and Speech 

Introduction to Political Science 

Issues in American Public Policy 
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American Legislative Process 
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Urban Sociology 

Elementary Spanish 
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SPAM 


201 


Intenrodiate ^>anlsh 




X W A 




SPAM 


303 


Advanced Graratar 


SPCH 


150 


Public Speaking 


TH 


370 


History of the Motion Picture 



following a coarse indicates a comrse misclassified according to the dis- 
criininant analysis of course clusters. 
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Figure b-13b* Cluster 2: Axgn posxtive mean resxauaxb uii mmxjf A«:Aiaw«A«4.M^ 
(ARE); high negative mean residuals on Antonyms (ANT). 



CLUSTER 


DEPT 


NUM 


D£SCRIPTIuri ni&\-Jj#i2>Dxr xvHXXLfri 








2 




zuz 


r T xnc xp xe s ox aw w wun uxng ± x 


2 


AC 


451 


federax mcoKie xoxai^xon ox xnaxvxuuoxs 


2 


ANTH 


102 


X n w rouilC u xOn x-o vai xx.ur ax #in wnrppo xoy y 


2 


ASTR 


101 


Descrxpiixve ABX-ronomy 


2 


ASTR 


lUZ 


iMscrxprxve ASwronoHiy 


2 


BA 


iiOQ 


buraxegxc nanageroenL./ roxxcy 


2 


BED 


450 


Ox X xce nacn ines x / isus xness nacnxnes x 


2 


BIO 


358 


MlcroDlology 


2 


BIO 


389 


Microbiology Lab * 


2 


CIS 


220 


Principles of Coioputer Progranndng I 


2 


CIS 


305 


AssCTjbly Language-Program&lng for Business Applications 


2 


CIS 


400 


Principles ot uj5U|>uter FrograBuning ii 




CIS 


410 


mtroouctlon to Data structures & ni-goruixsLXC Frocess 


2 


CIS 


434 


Data Nanagen^nt in Business 


2 


CIS 


4 DO 


systeois Anaxysxs ano x^sxgn 


Z 




46j 


Computer soxtwarei structure & organization 


z 




H fZ 


xnurcAjucnon uo uaraDase oysx-eins 


2 




4o0 


(A>Aipurer naraware 


2 


2\il 


iZl 


i/ecision naxne^iavxcs x 


2 


US 


/U 


uevexc^aiwnrax Keaaxng x 


2 


Ub 


c5U 


uevexopBffintLax &ngxisn x 


2 




1 Ail 

104 


vjoxxege Axgeora 


2 




ZUl 


int^rouucuion xo nanagesienu xnxonnat.xon oystieins 


2 


ILL 


ZUl 


Frxncxpxes ox nacroeconcmxcs 


*5 
Z 


VP 
ILL. 


ZUZ 


irrxncxpxes ox nxcroeconoiuxcs 


2 


ILL 




Money ana ureuix 


Z ' 


PUP 




mismess ftrxw-xng 


Z 


r 1 




uorporav-c r xnance 


Z 


f K 


lUl 


r<xe!nenu.ary rrencn 


Z 


r H 


lUZ 


£«xeaiieni*ary r rencn 


Z 




1 no 
XUZ 


E«xenieni.ary vjerman 


Z 


nirKLI 


101 


i>eginning Xfeisure Xiiie SKixxs 


Z 


INS 


3dU 


introduction to risk Management & Insurance 


z 


T PT C 


300 


Legal Environn^nt of Business 


z 


MATH 


211 


Calculus ot one Variable 


Z 


WAin 


Zlb 


intem^uiate calculus 


z 


WATH 


A A*^ 

44 7 


Metnocs oi Statistical Inrerence I 


2 


MATH 


44o 


Metnocs ox oxauxstiicai mterence ii 


z 


MV 


301 


Basic MarKeting 


z 


rniS 


237 


Mechanics 


Z 


FHiS 


Z Jo 


Execuricity » Magnetxsm 


z 




Z J9 


Heat / sound/ Lxgnt ^ Modern Fnysxcs ^ 


z 




101 


m troducx ion \x> Amer xcan vjoverninen l 


z 




A T 1 
4Z J 


Fsycnoxogy oi worK 


2 


RE 


ir\ 
JUi 


Real Estate Principles 


2 


RTP 


OC 
ZD 


n r xx. xng Kevxew i or une Kegen l s iLXani 


2 


SOC 


201 


Introduction to Sociology 


2 


SOC 


316 


Sex Roles in Modem Society 


2 


TH 


304 


History of World Theatre 



" following a course indicates a course misclassif led accord to the discrxjn- 
inant analysis of course clusters • 
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if\m.r-^ ft-ii*. Cluster 3: High positive mean residuals on Quantitative 
Figure 6-13c. Analytic Reasoning (J«E); high negative 

mean residuals on Reading Coaoprehension (RD) . 

=-====a=====«=-=-=^===« MTSCIASSIFICATIONS 



CLUSTER 


DEPT 


NUH 






3 


AC 


301 


3 


AC 


401 


3 


AC 


402 


3 


AC 


420 


3 


APPF 


100 


3 


BED 


456 


3 


BED 


471 


3 


CIS 


303 


3 


FED 


310 


3 


LGLS 


405 


3 


MATH 


220 


3 


MK 


434 


3 


MUS 


320 


3 


RMI 


350 



Basic Accounting Systems 
Financial Accounting I 
Financial Accounting II 
Managerial Accounting 
lulled Music-Introductory 
Beginning Typewriting 
Introduction to Word Processing 
AsseflOaly Language Progransning 
Educational Psychology 
Comprehensive Business Law 
Discrete Mathematics 
Sales Management 

Music for Elenentary Classroom Teacher 
Introduction to Risk Manaaeinent & Insurance 



following a course indicates a course misclassif ieJ according to the dis- 
criminant analysis of cout-e clusters. 



Figure 6-13d. Cluster 4: High positive mean residuals on Quantitative 

Coirparisons (QC). ^ 

ISSimor^^"^^ ^ ' MISCLASSIFICATIONS 

Federal Income Taxation I ^ 
Business Connminications 

Intro to Conputer-Based Information Systems Development 
Decision Mathematics II 
Intemwdiate German 
Intermediate German 

Intro to Data Structures & Algorithmic Process 
Entrepreneur ship & New Venture Management 
Introduction to Urban Life 

"•"'following a course'indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. ===== 



CLUSTER 


DEPT 


NUM 








4 


AC 


450 


4 


BED 


436 


4 


CIS 


210 


4 


DM 


122 


4 


GER 


201 


4 


GER 


202 


4 


IS 


410 


4 


MGT 


450 


4 


UL 


301 
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Figure 6- 



13e. Cluster 5: High positive rman residuals on Antonyms (ANT); 
high negative n»an residuals on Analytic Reasoning (ARE) and 
Reading Ccspreheneion (RD). 





DEPT 


HUM DESCRIPTION 


MISCLASSIFICATION 




AC 






it 






100 


mnirociucuion no fJiysACSx #wjiJirvpv>xogy 






nJT AT Jul 


ZUU 


AppX^0u nUBJLC IAjwBa /joVei 






ART 


inn 






5 


ART 


lUZ 






5 


ART 








5 


ART 


1 HA 






5 


ART 


1 nii 






5 


ART 


1 Tft 
X /D 


«9UJ>rvt3y vjx fmx u x 




5 


ART 


1 


OU^VClj w*. #1X V. XX 




5 


BA 




Tn^" v*r%/kir^^ 4 r>n Tnt*oiTiA^ 4nnn1 1<liiRinAflR 




5 


BIO 


111 
M X 


HiifnAn AnA^ofnv & PVivsinlonv 




5 


BIO 


112 


Hufnan Anatvoinv & PHvfiioloav 




5 


BIO 




UiimAn P^vfiioloiTrv 




5 




325 


Hiim&n Plivfi i^oloov X»ab 




5 


CHEM 


102 

X\J^ 


Oiraanlc & Hiodtewistirv Siirvev 




5 


CHEM 


111 

XXX 


pHom 4 r^fll Pt* \ np iol oe T 




5 


CHEM 


X X3 






5 


CHEM 


1 1 


OtiGm^s^xry 1j^]|^ X 




5 


ns 


fli 

a X 






5 


DS 






* 


5 


ENG 


91 9 






5 


FILM 


/ u 






5 


GEOG 




pnys icax beograpny x 




5 


GEOG 


1 n>i 


rnysxcBX ueoqropny xx 


• 


5 


HRTA 


o i n 
JIO 


HRT Personnel nanagement: 




5 


HRTA 


oon 


HRT Liaw 




5 


HRTA 


n Cn 


1 ravel lounsin 


♦ 


5 


IS 




Introduction to Data Procassing 




5 


JOUR 


1 ni 


Keporuxng x • x>hsxc journaixsui 




5 


MK 


«* J J 


rr mc xp xes ox s>ex x ing 




5 


MUS 


105 


Woodwind Laboratory 




5 


HUS 


193 


Introduction to Music 




5 


PHYS 


101 


Introductory Physics 




5 


PHYS 


210 


Physics for the Visual Arts 




5 


POLS 


320 


Coir^jarative Politics 




5 


PSY 


204 


Introduction to Applied Psychology 




5 


RTP 


25 A 


Writing Review for the Regent's Exam 




5 


SPAN 


101 


Elementary Spanish 




5 


TH 


410 


Contejnporary Theatre 


* 



following a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. 
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Figure 6 



OiUSTER 



I3f Cluster 6: High positive mean residuals on Analtyic Reasoning 

?iS?)rhigh n^aUve mean resiauals on Antonyms (ANT) and Reading 
Coo^rehens ion ( RD ) . 

DEPT NUM DESCRIPTION 

Hot available. 



MISCLASSIFICATIONS 



Q 




200 


D 


rur X * 


300 


£L 
D 




200 


t> 




300 


O 


ART 


466 


£L 
D 


niv 

on 


201 


D 




490 


D 




310 


£L 
O 




122 


D 




310 


D 




312 


D 




111 


D 




112 






305 


D 




102 


D 




345 


D 




309 


C 
D 




436 


D 


MATH 


126 


D 




430 


D 




435 


D 




436 


Q 




437 






439 


D 




470 


o 


MX 


410 


D 


MK 


420 


D 




430 


O 


MIC 


431 


D 


MK 


451 


£L 
D 


MK 


490 


f. 
O 




102 


D 


Mils 

n w w 


103 




MUS 

r* w w 


106 


6 


MUS 


108 


o 


MUS 


110 




MUS 

n w w 


12fe 




MUS 


144 


6 


MUS 


145 


€» 


MUS 


161 


& 


MUS 


191 




MUS 


244 


(> 


MUS 


245 


0 


MUS 


246 


& 


PHIL 


241 


& 


PSY 


101 


6 


PSY 


203 


6 


PSY 


301 



Not available. 
Applied Music-Lower Level 
lulled Music-Concentration 
Advanced Interior Design 

Introduction to Management Information Systems 

Seminar on Criminal Justice Problems 

Decision Mathematics III 

Survey of Calculus 

Introduction to Business Statistics 

General Modeling Techniques 

Composition I 

Coanpoaition II 

Human Growth & ttevelopment 

Historical Geology 

Physical Education for Elenantary Teachers 

Introduction to International Business 

Selection & Utilization of Education Media 

Analytic Geometry & Trigonometry 

Introduction to Personnel & Industrial Relations 

Organization Theory & Practice 

Personnel Selection S. Develojsnent 

Introduction to Organizational ConBminication 

CcHipensation Administration 

Operations Managen»nt 

Buyer Behftvior 

Marketing Research 

Advertising 

Advertising Campaigns 

Industrial Marketing 

Marketing Problems 

Voice Laboratory 

Not available. 

Instrumental Ensemble 

Choral Ensenble 

Concert Attendance 

Percussion Methods 

Musicianship 

Musicianship 

Music Survey 

Class Introduction to Piano 

Musicianship 

Musicianship 

Musicianship 

Introduction to Logic 

Contemporary Psychology^ 

Intro to Basic Psychological Process 

Psychological Statistics 



* 
* 
* 
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6 SPCH 101 Voice & Articulation 

6 SPE 401 Exceptional Children & Youth 



following a course indicates a course 
inant analysifi of course clusters. 



misclassified according to the discrim- 



. ,o ^i.,«4.-^ n- Hiah Kjsitive mean residuals on Analytic Reasoning 
Figure 6-139. ^J;^^«^^,-^«^Ji?:?t;t''co.^„ison» (QC); high negative »ean 



residuals on Antonyms (AST). 



sac; eras 

CLUSTER 



DEPT NUM 



7 


ART 


350 


7 


CJ 


371 


7 


HIST 


113 


7 


HIST 


476 


7 


JOUR 


201 


7 


JOUR 


302 


7 


JOUR 


304 


7 


JOUR 


306 


7 


JOUR 


410 


7 


JOUR 


421 


7 


JOUR 


450 


7 


JOUR 


454 


7 


JOUR 


498 


7 


PSY 


303 



DESCRIPTION 

Photogr^hy I 
Constitutional 



MI SCLASSIFICATIOHS 



liaw 



U.S. History 1492-Present 
History of the American City 
Reporting II: Hews Gathering/Reporting 
Hass Hedia & Society 
300 Years of Anwrican Journalism 
Journalism Law & Ethics 
Copyreading 

Business & Industrial Jotimalism 
Public Relations 

Cases & Problems in Public Relations 
Internship 

PSY 303 Principles/Methods of Psychological Investigation 

"•"'followinga'couise indicates a course misclassified according to the discrim- 
inant analysis of course clusters. 



Figure 6-13h. Cluster 8: High negative mean residuals on Analytic Reasoning 
(ARE) and Quantitative Comparisons (QC). 

,>^r>«, Mtiu nwqrPTPTTON MISCLASSIFICATIONS 

CLUSTER DEPT NUM DESCRIPTION 

Principles of Biology * 
Organisms & Their Environment 
Personal Growth & Self Development 
Theories of Personality 

"^"'following a course indicates a course misclassified according to the discrim- 
inant analysis of course clusters. 



8 


BIO 


141 


B 


BIO 


142 


8 


MH 


310 


B 


PSY 


416 



155 

139 - 
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Figure 6-13i. Cluster 9. High positive mean residuals on Reading CoBoprehension 
(RD), Analytic Reasoning (ARE), and Quantitative Caparisons (QC). 



CLOSTER 




MDM 


DESCRIPTION 


MISCLASSIFICAIK^ 


9 


BIO 


382 


Plant Biology 




9 


BIO 


384 


Animal Biology 




9 


BIO 


390 


Genetics 




9 


CHEN 


112 


Chemical Principles II 




9 


CHEM 


117 


Chemistry Lab II 




9 


CHEM 


118 


Chemistry Lab III 




9 


CHEH 


240 


Organic Chonistry I 




9 


CHEM 


241 


Organic Chranistry II 




9 


CHEM 


242 


Organic Chemistry III 




9 


CHEN 


460 


Bicx:heanistry I 




9 


EHG 


315 


Introduction to Creative Writing 


* 


9 


MATH 


212 


Calculus of One Variable 


* 


9 


MATH 


215 


Multi-variable Calculus 




9 


MATH 


335 


Matrix Theory 




9 


MATH 


435 


Linear Algebra 




9 


MATH 


451 


Mathematical Statistics I 




9 


MATH 


461 


Advanced Calculus I 




9 


MATH 


462 


Advanced Calculus II 




9 


MGT 


350 


Management Concepts Theory & Practice 


* 



•**" following a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. 



Figure 6-13 j. Cluster 10. High negative mean residuals on Quantitative 
Coiparisons (QC). 



CLUSTER 


DEPT 


NUN 








10 


CHEM 


101 


10 


DS 


50 


10 


DS 


71 


10 


ENG 


385 


10 


MATH 


107 


10 


MATH 


122 


10 


MH 


498 


10 


PSY 


105 


10 


PSY 


202 


10 


PSY 


356 


10 


PSY 


358 


10 


PSY 


404 



DESCRIPTION MISOASSIFICATIONS 



General Chemistry 

Personal 6 Acad^nlc Develoinnent Seminar * 
Develoimiental Reading II * 
Southern Literature 

Eleirontary Statistics I * 
Not available. 
Not available. 
Foundations of Psychology 

Introduction to Social/DeveloiMnental Psychology 
Leadership & Group Dynamics 
Psychology of Human Potential 
Child Development 

following a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters- 




Figure 6-13k. Cluster 11. High positive mean residuals on Analytic Heasoning 
(ARE) and Quantitative Coaoparisons (QC) . 



CLUSTER DEPT NUM DESCRIPTIOH MISCIASSmCATI<»i8 

11 CJ 301 Introduction to Criminal Justice 

11 CJ 311 The American Police System 

11 CJ 321 Juvenile Delinquency 

11 CJ 331 Corrections 

11 CJ 370 Courts & Basic Criminal Procedure 

11 CJ 411 Principles of Investigation 

11 CJ 475 Criminal Law 

11 CJ 494 Criminal Justice Field Instruction 

11 DS 92 Introduction to Algebra 

11 ENG 201 Europ^n Literature I * 

11 GEOG 101 Introduction to Hunan Geogre^hy * 

11 US 301 Introduction to Urban Studies 

11 US 302 Methods of Urban Research 

"*" following a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. 



Figure 6-131. Cluster 12. High positive mean residiials on Reading 
Con^rehension (RO). 



CLUSTER 


DEPT 


NUM 


DESCRIPTION 


Ml SCLASSIFICATIONS 












12 


DM 


312 


Not available. 




12 


ENG 


113 


Advanced Coirposition 




12 


ENG 


211 


Survey of English Literature I 




12 


ENG 


409 


History of the English Language 




12 


ENG 


435 


Shakespeeore: Tragcidies 




12 


GEOG 


350 


Urban Geography 




12 


GER 


101 


Elementary Ctorman 




12 


MATH 


105 


Math for Liberal Arts 




12 


MATH 


125 


Not available. 




12 


PHIL 


302 


History of Western Philosophy II 




12 


PHYS 


102 


Introductory Physics 




12 


POLS 


414 


Urban Politics 




12 


POLS 


462 


Not available. 




12 


PSY 


201 


General Psychology 




12 


PSY 


314 


Abnormal Psychology 


* 


12 


RUS 


101 


Eleinentary Russian 




12 


RUS 


102 


Elementary Russian 




12 


RUS 


201 


Xnteni»diate Russian 




12 


3PCH 


445 


Nonverbal Communicaticn 





follwing a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. 




Figure e-^lSin* Cluster 13. High negative n^an residuals on Analytic Reasoning 
(if^) and Quantitative Comparisons (^)* 



CLUSTliF :?En HUM DESC31IPTI<» MISOASSIFICATIONS 



13 MATH 104 College Algebra 

13 MGT 401 Introduction to Organizational Behavior 

13 SOC 308 Sociology of the Family 

«*«' following a course indicates a course misclassif ied according to the discrim- 
inant analysis of course clusters. 
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VII. Sunmarv and Conclusion 



This report concentrated on an examination of transcripts and assessment 
test scores from the conibined native Bangle of graduating seniors at Georgia 
State University in the 1986-87 and 1987-88 academic years. The main purpose of 
this project was to determine if enrollment in different patterns of coursework 
were associated with gains in the general learned abilities of \indergraduate 
students. The answer to this question was consistently "yes". Roughly 8.1 of 
10 courses analyzed were accurately grouped according to differential effects in 
the general learned abilities of students. Taking different patterns of 
coursework did lead to different types and levels of development as measured by 
the nine item-types of the GRE General Test. 

Several consistent findings emerged frran the analysis of coursework 
clusters. First, the development of general learned abilities did not have an 
exact one-to-one relationship with departmental categories. All quantitative 
reasoning develoiwnent did not occur exclusively in Mathematics -related courses, 
for exan?>le- Consequently, simple counts of the nunber of credits or courses a 
student has taken in a particular subject may not be a reliable proxy of general 
learning in the attendant subject area. Quantitative skills may be developed in 
a variety of courses such as accounting, manageanent, and ccaoputer information 
systems. Second, the 'aveloinnent of general learned abilities was not confined 
to the lower division. While there is clearly a difference between general 
education and general learning (as measured by the GRE), general education 
requirements should be re-examined in light of student improvement in general 
learned abilities- Coursework taken by students who shwed significant gains 
yhould be examined, evaluated and incorporated into the general education se- 
quence of the college- Third, beyond the college catalog, there was little 

- 
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formal monitoring and description of the curriculum in terms of general learned 
abilities at the college-wide or university-wide level. Colleges should 
regularly monitor the n\inber of credits and courses in their curriculum. 
Without this baseline data, the extent to which students share a ccsBoon learning 
experience at a college cannot be readily determined. 

The relationships established through the cluster analytic model were asso- 
ciational, not causal. Once a set of courses has been linked to inqprov^nnt in 
a specific learned ability, a targeted investigation can be launched to deter- 
mine the cca!w»nalities of teaching- learning environxront, of student and faculty 
expectations of performance, of the specific abilities of the students who en- 
rolled in the classes- But regardless of what hypotheses are generated about 
why this coursework/programs are associated with gains in learned abilities^ one 
can state with confidence that students who enrolled in this coursework 
demonstrated gains on a specific type of learned ability. 

Coursework patterns with negative means have limited meaning, h negative 
mean of residuals on a coursework cluster does not necessarily mean actual 
decline in general learning, only decline about the group tested. Regression 
automatically defines half the residuals as negative. The mean performance of 
the group tested is the basis upon which the individuals* GRE item-type scores 
are predicted front the corresponding SAT scores. By definition, half the group 
falls below the mean, half is above it- Therefore, those with negative means 
may have gained in general learned abilities. The negative sign shows that 
t>ieir gain fell below the n»an of students in the sanple. Declines in general 
learned abilities are relative to the sannple group and may or may not represent 
actual declines in abilities. 

Gains in student learned ability may be attributable to learning outside 
the classroom. Programs in this research are the settings for analysis. 



Con^iarabXe analysis could be conduct»d according to student residential 
groupings. It is known, for exanple, that on many caa^^ee the acadoiic 
iwrformance of students in one fraternity or sorority nay be consistently above 
the average for that college, while students living at another Greek residence 
setting may be well below the caofnis norms for acadesoic performance. Does 
living in a fraternity or sorority cause higher or lower academic performance? 
Not necessarily. These are selective and self -selected residential situations; 
the relationship with academic performance is associational . 

The coursework patterns identified in this research include general 
education, major, minor and electives. No a priori distinction was made 
according to these categories prior to analysis of the data. A physical 
education course in Tennis may be associated with isiproved learning in Regular 
Mathematics. This does not mean that enrolling in Tennis causes improved 
abilities in Regular Mathematics. What it does mean is that students who 
enrolled in this course tended to improve in this learned ability. Why? The 
cluster analytic model does not tell us why. Its purpose is to sort through the 
hundreds or thousands of courses in the college curriculum. The model points 
out those coursework patterns taken by students who inprove in general learned 
abilities. 

The cluster analytic model of analysis is admittedly coo^jlex. It would be 
siji^Jler to calculate residuals on one iteaa-type, such as Regular Mathematics, 
and then to rank order all coursework according to the mean residuals of 
students in each course. This would give a picture of each course according to 
one measure of general learning. However, it would not givi. an idea of the role 
of that strength of that ability or measure relative to o. her measures of 
general learning used in the assessnwnt. The discriminant analysis shows the 
role of various types and measures of general learning relative to the 
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coursework that students take in college. 

Where does this leave us? First, we ackiuwledge Uiat the cluster analytic 
Btodel perforMd well with the conbined sai^le at Georgia State University. It 
sorted and classified programs according to a given set of n^asures of general 
leaxned abilities « Second^ sane program patterns identified snacte sense. &sbb 
programs involving mathematics were associated with gains in mathoDatics 
abilities. 

If all student coursewrk was distributed randomly throughout the 
curriculum^ so too would be their test score residuals. A non-randOTS 
distribution of residuals inplies that specific coursework makes a difference in 
the developront of general learned abilities. Only those courses selected by 
students showing iisprovement above the mean for their group should have positive 
mean course residuals. This research affirmed the differential courseirark 
hypothesis. This research also affirmd the person environment fit hypothesis 
(Pascarella, 1985). 

Student need not choose frc^n several hundred or several thousand courses to 
have an effective education. The existence of a multitude of courses may be a 
healthy sign. It inay show that colleges and universities accurately mirror the 
explosion and ccm^lexity of knowledge in the last decade of the twentieth 
century. Such ccar?>lexity need not jjiqmir the development of general learned 
abilities in undergraduates. 

Yet, this research suggests that many students do not make wise course 
selections. At least they don't make smart choices in relation to those general 
learned abilities tested by the most commonly recognized post-baccalaureate test 
of general learning- The mean residuals of most courses taken by students in 
common with other students of the same cohort was near zero. This means that 
the general effect of coursework on student learning was randcOTly distributed 



across the curriculum. Perhaps only one-third of coursework talran in coamon (5 
or more students) could be found to have a positive relationship with general 
learning as measured. This rate of return can be improved by developing more 
discrete arrays of coursework a{5^1icable to general education requireiMints and 
by organizing and info:.ming the student academic advising prcK:ess so that 
students may choose arong coursework aligned with their abilities and prior 
leamirg* To that end, the cluster analytic model for identifying the 
differential effect of coursework holds promise to revising and enhancing the 
conditions for excellence in undergraduate general learning. 

It is ccaxsK^nly assumed that the general curriculum leads to gains in 
general learning and the u^jor^ minor and elective curriculum leads to gains in 
subject area learning* If such were the case, then the positive residuals on 
measures of general learning should occur with the courses which students hold 
in cc^nmon. Also, negative mean course residuals should be primarily distributed 
anK^ng courses which students did not hold in cCTBOon. 

Vie know a great deal about what colleges say should be the goals and 
stanaards for a baccalaureate degree. The DCP research suggests that much 
future research is needed to determine what curricular patterns and trends 
consistently produce the gains in general learning that institutions seek to 
impart to their students. The challenge of understanding the specific intact of 
coursework on the learning of students has just begun. 
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